A recurrent neural network reads data one step at a time, like reading a sentence word by word, and carries a running memory forward. Plain recurrent networks tend to lose track of things that happened many steps earlier. An LSTM adds a separate memory track plus “gates,” small learned switches that decide what to write into memory, what to erase and what to read out. That lets it connect “The keys that I left on the kitchen table this morning” with “are” many words later.
LSTMs were introduced by Sepp Hochreiter and Jürgen Schmidhuber in 1997. Through the mid-2010s they powered speech recognition, handwriting recognition and early neural machine translation.
Why it matters
Transformers have since replaced LSTMs in most language models, because attention handles long context better and trains faster on modern hardware. LSTMs are still useful history for understanding why transformers mattered, and they remain in use for some time-series and on-device tasks.
No published Chain of Thought episode discusses LSTMs directly yet, so this entry relies on the sources below.
Related terms
Recurrent neural network is the broader family. Attention mechanism is the idea that displaced LSTMs in language models.
Sources
- Goodfellow, Bengio and Courville: Deep Learning, Chapter 10 (Sequence Modeling) — Textbook chapter explaining recurrent networks, the long-term dependency problem and gated units including the LSTM; checked October 24, 2026.
- Greff et al.: LSTM: A Search Space Odyssey — Large-scale study of LSTM variants that describes the standard LSTM's input, forget and output gates and credits its 1997 origin to Hochreiter and Schmidhuber; checked October 24, 2026.