Post-training

A base model from pre-training continues text; it doesn’t reliably answer questions or follow instructions. Post-training changes that. It usually starts with supervised fine-tuning on examples of good conversations, then preference training, where the model learns from human or AI rankings of its answers (RLHF or DPO). Newer reasoning models add reinforcement learning on problems with checkable answers, like math and code.

Adapting an existing model this way is usually much cheaper than pre-training one, though large reinforcement learning runs can be expensive too. That is why companies now do their own post-training on top of open models. On Chain of Thought, Intercom described post-training a small open model to take over one high-volume task from a frontier model, and Thomson Reuters described post-training an open model on its legal expertise. How large language models actually work explains where post-training fits.