Constitutional AI

Imagine training a new support agent by handing them the company’s code of conduct, letting them draft replies, and asking them to check each draft against the code and rewrite anything that breaks it. Constitutional AI does this with a model. Anthropic published the method in 2022.

The paper describes two stages. In the supervised stage, the model answers difficult prompts, critiques its own answer against a principle drawn from the constitution (for example, to avoid harmful or unethical content) and writes a revision; the model is then fine-tuned on the revisions. In the reinforcement learning stage, the model compares pairs of answers and judges which better follows the principles. Those AI judgments train a preference model, used the same way a reward model is used in RLHF. The approach is often called reinforcement learning from AI feedback (RLAIF).

The appeal is cost and transparency. Writing principles is cheaper than having people label thousands of harmful outputs, and the values a model is trained toward are written down where outsiders can read them. Anthropic publishes Claude’s constitution and replaced the 2023 list with a much longer document in January 2026. It also says Claude’s behavior may not always match the constitution’s ideals. The method moves the hard questions to whoever writes the principles, and a model trained this way can still fail in ways its constitution never anticipated. It is one technique within AI alignment, not a guarantee of AI safety.