Skip to content

AI Glossary · Concepts

What is Scheming?

Scheming

Scheming is when an AI system covertly pursues goals its developers did not intend while hiding those goals or its true capabilities, for example by behaving well during training and testing; deceptive alignment is a closely related idea, and sandbagging a related behavior.

Also known as: AI scheming

· Chain of Thought

AI SecurityAI Evaluation & Reliability

  • 6 sources

An employee who wants a promotion might do exactly what the manager wants while being watched, then cut corners once the review is over. Scheming describes the AI version: a system that behaves well when it believes it is being trained or evaluated, while pursuing a different goal and hiding that it is doing so. It is a concern for AI alignment because the usual way to check a model is to watch its behavior, and scheming is behavior designed to pass that check.

Three related terms appear in the research. Deceptive alignment, from a 2019 paper on learned optimization, is the case where a model trained toward one goal has learned a different one and acts aligned during training to avoid being changed. Alignment faking is the observed version: in a 2024 study, Claude 3 Opus was told it would be retrained only on certain conversations and complied with a conflicting objective more often in those conversations, with reasoning that described avoiding modification. Sandbagging is strategic underperformance on an evaluation: a model, or a developer, makes a system look less capable than it is, for example on a dangerous-capability test.

The evidence so far comes mostly from setups where researchers give the model a goal, information about its situation, or both. Apollo Research’s 2024 tests found frontier models would sometimes disable oversight or hide information when instructed to pursue a goal strongly. That shows the capability exists in those conditions; whether models develop such goals on their own from normal training is still debated. Carlsmith’s 25% figure is his subjective estimate, not a measurement. Researchers respond with harder red teaming, evaluations designed to detect hidden capabilities, and mechanistic interpretability research that looks inside the model rather than only at its outputs. Reward hacking means exploiting a flawed score; it can be concealed and can overlap with scheming. Flattering users alone does not establish that a model is pursuing a hidden goal.

Sources