Say a clinic has only 500 labeled patient records. Holding back 100 as a single validation set leaves less to learn from, and the score depends heavily on which 100 were picked. Five-fold cross-validation splits the records into five groups of 100. The model is trained five times, each time on four groups and scored on the fifth, and the five scores are averaged. Every record gets used for both learning and checking, just never in the same round.
The average is a steadier estimate of generalization than one split, and the spread between folds shows how much the result could swing. Teams use it to compare models or hyperparameter settings fairly. It does not replace a final test set: if you pick the winner by cross-validation score, you still want an untouched set to confirm it.
The folds have to respect the data’s structure. scikit-learn’s guide recommends group-wise splits when many rows come from the same person or device, so the same patient never lands on both sides. For time series, shuffled folds let the model train on the future and test on the past; time-series splits keep training data earlier than test data (see time-series forecasting). Preprocessing such as feature scaling should be fit inside each fold, otherwise the held-out fold leaks into training (data leakage). Cross-validation is common for classic models on small and medium datasets; it is rarely used to train large language models, where a single run can cost far too much to repeat five times.
Sources
- scikit-learn: Cross-validation: evaluating estimator performance — Explains k-fold cross-validation (train on k minus 1 folds, test on the remaining fold, rotate), group-wise splits, and why shuffled folds give unreasonable results on autocorrelated time series; checked October 24, 2026.
- Google Machine Learning Glossary: cross-validation — Defines cross-validation as estimating generalization by testing against one or more non-overlapping subsets withheld from the training set; checked October 24, 2026.