Imagine a news site predicting which headlines readers will click. Reader interests change hour by hour, so instead of retraining from scratch every month, the model updates its weights after each new batch of clicks. That is online learning. The word “online” refers to learning from a stream of data, not to the internet, and it is different from simply running a trained model in production, which is inference.
scikit-learn calls this incremental learning: estimators that implement partial_fit can learn from one mini-batch at a time, which the docs note is sometimes called online learning.
Why it matters: online learning lets a model keep up with concept drift (changes in the patterns the model learned) and with datasets too large to hold in memory. The risks are that a bad batch can pull the model off course and that new data can overwrite older lessons, the problem known as catastrophic forgetting. Most large language models are not trained this way after release; they are updated through separate training runs or fine-tuning. That last point is editorial context, not a claim from a Chain of Thought episode.
Sources
- scikit-learn: Strategies to scale computationally (incremental learning) — Official docs; supports incremental learning with partial_fit and its description as "online learning", checked October 24, 2026