Further training can improve a new task while damaging earlier capabilities. Kirkpatrick and colleagues studied this problem and proposed elastic weight consolidation, which constrains updates to weights important for previous tasks (Overcoming catastrophic forgetting in neural networks). That is one mitigation studied in particular experiments, not a promise that any adapted language model will retain everything it could do before.
For example, imagine adapting a general assistant to legal text. Its legal terminology might improve while its ability to follow a JSON format or solve a routine coding question gets worse. Build a retention set before training: include the formats, reasoning tasks and non-legal questions your product must still handle. Run the same set against both the starting and adapted checkpoints.
Discussing the coding, maths and general capabilities Thomson Reuters wants to preserve, Joel Hron says, “when we apply our data to it, we don’t want to lose those capabilities.” (episode 69, 9:47).
For builders, do not reduce evaluation to the new specialty’s score. AWS’s data-mixing guidance describes combining old and domain data and evaluating both domain and general benchmarks. Continued pre-training covers the adaptation stage, while model training connects the methods. How to test an AI system covers turning required behavior into checks.
Sources
- Overcoming catastrophic forgetting in neural networks — Primary source for the explanation and distinctions in this entry.
- data-mixing guidance — Primary source for the explanation and distinctions in this entry.
