Supervised Fine-Tuning
Supervised fine-tuning (SFT) trains a pretrained model on examples of desired behavior, commonly inputs paired with target responses. It updates model parameters so the model becomes more likely to produce responses like those examples.
Also known as: SFT
A support team might collect questions with carefully reviewed answers, then train a model to follow that response style and task behavior. The examples are training data. Putting the same examples in a prompt instead is few-shot prompting, which does not update the weights.
SFT is one form of fine-tuning. Preference optimization uses comparisons or preferences, while reinforcement learning optimizes a reward signal. These approaches can occur in the same post-training pipeline.
Example: hold out recent, representative requests before training a query-rewriting model. Evaluate whether it preserves identifiers and user intent rather than merely imitating the formatting of the training answers. Fergal Reid discusses Intercom’s post-training choices in episode 49. That example supports treating training as an experiment with a baseline, not an automatic next step for every AI application.
Sources
- Hugging Face TRL: SFT Trainer — Documents supervised fine-tuning datasets, loss, and training choices.
Go deeper
- Hugging Face: Supervised fine-tuning course course
Follow dataset preparation and a training example, with validation and overfitting checks.
