A training run for a large model can take weeks on thousands of chips, and hardware fails. Without saved progress, one crashed machine could mean starting over. So training code periodically writes a checkpoint, like saving a video game: the model’s parameters and, for resuming, the optimizer state and the current step or epoch. PyTorch’s guide shows that saving only the weights is enough to run the model, while resuming training needs the extra state.
Checkpoints are also how teams choose which version to keep. Scores on the validation set often peak before training ends, because later steps start overfitting, so the best checkpoint may come from the middle of the run; keeping it is the idea behind early stopping. Researchers study how capabilities appear across checkpoints, and fine-tuning starts from someone else’s released checkpoint rather than from scratch.
In everyday AI talk, a “checkpoint” is often simply a downloadable model file: when a lab releases open weights, it publishes checkpoints. Agent and coding tools also use “checkpoint” for saved conversation or file states you can roll back to, a separate idea, which is why this entry autolinks only the model-specific phrases.
Sources
- Google Machine Learning Glossary: checkpoint — Defines a checkpoint as data capturing a model's parameters during training or after it completes, used to save weights across sessions, resume after errors and compare training stages; checked October 24, 2026.
- PyTorch: Saving and loading a general checkpoint for inference and/or resuming training — Shows that a resumable checkpoint stores the model state_dict plus the optimizer state, epoch and latest loss; checked October 24, 2026.