A support team might collect questions with carefully reviewed answers, then train a model to follow that response style and task behavior. The examples are training data. Putting the same examples in a prompt instead is few-shot prompting, which does not update the weights.
SFT is one form of fine-tuning. Preference optimization uses comparisons or preferences, while reinforcement learning optimizes a reward signal. These approaches can occur in the same post-training pipeline.
Example: hold out recent, representative requests before training a query-rewriting model. Evaluate whether it preserves identifiers and user intent rather than merely imitating the formatting of the training answers. Fergal Reid discusses Intercom’s post-training choices in episode 49. That example supports treating training as an experiment with a baseline, not an automatic next step for every AI application.
Sources
- Hugging Face TRL: SFT Trainer — Documents supervised fine-tuning datasets, loss, and training choices.
Go deeper
- Hugging Face: Supervised fine-tuning course course
Follow dataset preparation and a training example, with validation and overfitting checks.
- Scaling Instruction-Finetuned Language Models paper
Read the study's comparisons of instruction-tuning task count, model size and chain-of-thought training data.