Imagine grading a stack of 10,000 practice exams to work out what to teach differently. You could change the lesson after every single exam, which is noisy and slow. You could read all 10,000 first, which takes forever per change. Or you could read 100 at a time, adjust, and continue. That middle option is a mini-batch, and 100 would be the batch size.
During training, the model processes one batch, measures its average error, and makes one update to its parameters. An epoch is one full pass over the training data, so with 10,000 examples and a batch size of 100, an epoch is 100 updates. Like the learning rate, batch size is a hyperparameter chosen by people.
The choice has practical costs. Bigger batches use the parallel hardware of a GPU more efficiently but need more memory, and research such as Keskar et al. found very large batches can hurt how well a model generalizes unless other settings change too. Goyal et al. showed one fix: scale the learning rate along with the batch and warm it up gradually. So when a team says it “doubled the batch size,” expect the learning rate and run time to change as well. At inference time, “batching” means grouping user requests to serve them efficiently; see continuous batching for that separate idea.
Sources
- Google Machine Learning Glossary: batch size — Defines batch size as the number of examples processed per iteration and contrasts stochastic (batch size 1), full-batch and mini-batch strategies, calling mini-batch usually the most efficient; checked October 24, 2026.
- Keskar et al.: On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima — Reports that very large batches have been observed to degrade a model's ability to generalize, compared with the 32 to 512 example batches common at the time; checked October 24, 2026.
- Goyal et al.: Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour — Shows large mini-batches can train an image model accurately when the learning rate is scaled with batch size and warmed up, spreading one batch across many GPUs; checked October 24, 2026.