ARC-AGI

Each ARC-AGI-1 or ARC-AGI-2 task shows a few pairs of small colored grids: an input and the output it should become. The solver has to work out the rule, perhaps “fill every enclosed shape with the color of its border,” and apply it to a new input. Most people can solve many tasks with a little thought. The tasks are designed so that memorized facts don’t help; every puzzle needs a rule the solver has not seen before.

François Chollet introduced the benchmark in 2019 with an argument about what intelligence is: not how much a system knows, but how efficiently it picks up a new skill from little data. That made ARC-AGI a frequently cited test in debates about artificial general intelligence. Progress on it by reasoning models is closely watched for that reason. The ARC Prize Foundation runs competitions and an official leaderboard, and released harder versions: ARC-AGI-2 in 2025, and ARC-AGI-3, which tests whether a system can adapt on the fly to new interactive environments.

Scores need careful reading. The foundation’s leaderboard plots cost per task against performance, because spending a great deal of compute on each puzzle can raise a score in a way that says little about efficient learning. And solving puzzles is one narrow measure: doing well on ARC-AGI is not the same as general intelligence. Like any AI benchmark, it measures one ability well.