A base model from pre-training continues text; it doesn’t reliably answer questions or follow instructions. Post-training changes that. It usually starts with supervised fine-tuning on examples of good conversations, then preference training, where the model learns from human or AI rankings of its answers (RLHF or DPO). Newer reasoning models add reinforcement learning on problems with checkable answers, like math and code.
Adapting an existing model this way is usually much cheaper than pre-training one, though large reinforcement learning runs can be expensive too. That is why companies now do their own post-training on top of open models. On Chain of Thought, Intercom described post-training a small open model to take over one high-volume task from a frontier model, and Thomson Reuters described post-training an open model on its legal expertise. How large language models actually work explains where post-training fits.
Sources
- Ouyang et al.: Training language models to follow instructions with human feedback — Describes the supervised fine-tuning then RLHF recipe, and reports that a 1.3B InstructGPT model was preferred to the 175B GPT-3 by human raters; checked October 25, 2026.
- Rafailov et al.: Direct Preference Optimization — Introduces DPO, which trains on human preference pairs directly instead of fitting a separate reward model and running reinforcement learning; checked October 25, 2026.
- DeepSeek-AI: DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning — Uses large-scale reinforcement learning to develop reasoning, with the strongest gains on verifiable tasks such as mathematics and coding competitions; checked October 25, 2026.
Go deeper
- How do large language models actually work? AI, decoded · How Large Language Models Actually Work
- Should you use prompting, RAG, or fine-tuning to customize an AI model? AI, decoded · Fine-Tuning vs. RAG vs. Prompting: How to Choose
- When should you use a small language model instead of a frontier model in production? AI, decoded · Small Language Model vs. Frontier Model
- Tulu 3: Pushing Frontiers in Open Language Model Post-Training paper
Study one open pipeline spanning data curation, supervised fine-tuning, DPO and reinforcement learning with verifiable rewards.
From the conversation
-
How Intercom Cut $250K/Month by Ditching GPT for Open Models | Fergal Reid -
Thomson 1: The New $40M Legal AI Model | Thomson Reuters Joel Hron -
Agents Are Overtaking Humans in Search: Why Search Must Be Rebuilt -
Most of the Web Will Never Get APIs for AI Agents | Dhruv Batra, Yutori -
Beyond Transformers: How Liquid AI Is Rethinking LLM Architecture | Maxime Labonne -
The AI Framework Era Is Over: Why Context Is the Moat | Jerry Liu of LlamaIndex -
For AI Agent Fleets, the Model Is Rarely the Bottleneck | Theory Ventures' Andy Triedman -
11 Cameras, Dozens of Mics: The AI That Reads the Room | Tormod Ree, Neat