Fine-Tuning

Fine-tuning starts from a pretrained model and adjusts its parameters using additional examples. Hugging Face’s training guide describes this process and walks through preparing a dataset and training a model. Fine-tuning is one form of transfer learning: knowledge from pre-training is reused for a narrower task.

The distinction from prompting and RAG is where the change lives. A prompt or retrieved document supplies information at inference time. Fine-tuning changes the model parameters used to produce the response. These approaches can be combined.

As an engineering starting point, consider fine-tuning when you have a repeated task, suitable training examples, and an evaluation set that can establish whether the change helps. Compare quality, training effort, and serving cost against your existing approach; a smaller specialized model is not automatically cheaper or better once those costs are included.

Fine-tuning can teach domain information as well as behavior. It does not provide a live connection to changing records. If a task depends on the latest customer balance or policy document, supplying current evidence through retrieval is often easier to update and inspect than retraining for every change.