Skip to content

AI Glossary · Concepts

What is Ablation?

Ablation

Ablation is an experiment that removes or disables part of a system and measures how the result changes. Controlled comparisons help estimate what that component contributes.

Also known as: ablation study

· Updated · Chain of Thought

AI Evaluation & Reliability

  • 1 source

A team evaluating a RAG assistant might compare retrieval plus reranking with retrieval alone. Keeping the generator, questions and scoring rules fixed helps reveal whether the reranker improves answers enough to justify its latency. This is an illustrative experiment, not a reported result.

Removing a component measures its contribution within that particular system. It does not show that the component is useless in every design: other parts may compensate for it, or depend on it. Changing the dataset and several components at once makes attribution harder.

State whether each variant is retrained or retuned. Disabling part of a fixed model and training a replacement without it answer different questions. Report the evaluation conditions and variation across runs so a small apparent gain does not become an unsupported architectural claim.

Remove components one at a time and together

Extend the retrieval example to two optional components, query rewriting and reranking. Each variant answers the same 100 questions with the same pass criterion:

Query rewritingRerankingPassing answers
OnOn90
OffOn89
OnOff88
OffOff70

Removing rewriting alone costs one point and removing reranking alone costs two, yet removing both costs 20. Each component covers for the other here, so the single removals understate what the pair contributes, and dropping both because each looked cheap would be a mistake. Run the pair as well as each single removal, compare the questions each variant fails as well as the totals, and repeat runs when outputs vary before reading meaning into a one-point gap.

Sources

Go deeper