Training is a loop: show the model examples, measure its error with a loss function, nudge its parameters, repeat. Plotted over time, the loss usually drops steeply, then flattens. When it has flattened and stays flat, the run has converged. Think of a ball rolling into a valley: eventually it stops moving much.
Teams watch these loss curves to decide when to stop and to spot trouble. A curve that oscillates or shoots upward usually points to a learning rate that’s too high, a batch size problem, exploding gradients or bad data. Google also warns about a false finish: loss can plateau for a while and then start falling again, so a short flat stretch is not proof of convergence.
Converged doesn’t mean good. A model can converge to a poor solution (underfitting) or settle while memorizing the training data, which is why the validation loss is checked alongside the training loss (overfitting). Large language models are often stopped on a compute budget long before their loss fully levels off, and scaling laws are used to predict where the curve is heading. The ordinary-English word “convergence” is common, so this entry only autolinks training-specific phrases.
Sources
- Google Machine Learning Glossary: convergence — Defines convergence as the state when loss values change very little or not at all with each iteration, and warns a loss can stay flat for a while before falling again; checked October 24, 2026.
- Google Machine Learning Crash Course: Interpreting loss curves — Walks through oscillating, exploding and diverging loss curves and their usual causes, such as a learning rate that is too high or bad data; checked October 24, 2026.