The core idea is an if-then commitment. A lab decides in advance which capabilities would be dangerous, for example meaningfully helping someone build a biological weapon or carry out autonomous cyberattacks, then runs evaluations on each new model. If a model crosses a threshold, stronger security and deployment safeguards must be in place before it ships, or the model is not deployed. Anthropic calls its version a Responsible Scaling Policy, first released in September 2023; Google DeepMind’s Frontier Safety Framework uses “Critical Capability Levels” across autonomy, biosecurity, cybersecurity and machine learning R&D.
These policies spread quickly. At the AI Seoul Summit in May 2024, sixteen companies signed the Frontier AI Safety Commitments, which include publishing such a framework; METR, which tracks the policies, reports that four more companies joined later. The policies differ in thresholds and enforcement, but share the pattern of capability evaluations, thresholds, safeguards and governance.
For buyers and builders, these documents are the best public signal of how a lab will handle a dangerous frontier model. Editorially, they are self-imposed: the company writes, interprets and can revise its own policy, which is why outside testing by AI safety institutes and red teaming matter. They are a specific instrument within broader AI governance.
Sources
- Anthropic: Responsible Scaling Policy — Says Anthropic released the first version in September 2023; page last updated August 14, 2026; checked October 24, 2026.
- Google DeepMind: Introducing the Frontier Safety Framework — Defines Critical Capability Levels, early-warning evaluations and mitigations, with initial domains of autonomy, biosecurity, cybersecurity and machine learning R&D; checked October 24, 2026.
- GOV.UK: Frontier AI Safety Commitments, AI Seoul Summit 2024 — Commitments published May 21, 2024 and updated February 7, 2025 with additional signatories; checked October 24, 2026.
- METR: Common Elements of Frontier AI Safety Policies — Compares published policies; reports sixteen companies agreed at the Seoul summit to publish them, four more joined later, and twelve had published policies when the page was written; checked October 24, 2026.