The concept behind the show
AI reasoning, explained
Chain of Thought takes its name from a sequence of intermediate reasoning steps leading to an answer. Explore how that idea appears in prompting and reasoning models, what extra computation can change, and how to evaluate the result. The glossary and conversations below cover the vocabulary and engineering choices.
- 4 core terms
- 6 conversations
- 3 common questions
How the idea evolved
-
Show the work2022
Chain-of-thought prompting uses instructions or examples to elicit intermediate steps before an answer. In the 2022 study by Wei and colleagues, examples containing reasoning steps improved arithmetic, commonsense and symbolic reasoning results for the large language models tested. Those benchmark results do not establish that the same prompt will improve every model or task.
-
Spend more computation answeringTest-time compute
Reasoning models are trained to use additional computation when producing an answer. Some offer a choice: at its February 2025 launch, Claude 3.7 Sonnet supported both ordinary responses and extended thinking. Test-time compute also includes methods for searching and checking candidate answers. In Snell and colleagues’ research, prompt difficulty affected which approach used that compute most effectively.
Training time A bigger modelorTest time More computation while answering -
Know the limitsMeasure on your workload
A written chain of reasoning can omit information that influenced an answer. In an April 2025 study, Claude 3.7 Sonnet and DeepSeek R1 often omitted hints they had used. Check the final answer, including factual claims that could be AI hallucinations. Measure quality, cost and latency on your workload before choosing how much computation to spend.
Prompting vs. reasoning models
The same idea, in two places: in your prompt, or inside the model.
| Aspect | Chain-of-thought prompting | Reasoning model |
|---|---|---|
| How you get it | Ask for intermediate steps, or give examples that contain them | Trained to use additional computation; some models make extended thinking optional |
| What it does | Elicits intermediate steps before the answer | Spends additional computation while producing the answer |
| The lever | The prompt | Test-time compute: more computation while answering |
| Both | Results depend on the model, task and method, and more computation costs latency and tokens. A written chain can omit what influenced the answer, so check the result. | |
The vocabulary
- Chain-of-Thought Prompting Chain-of-thought prompting supplies or requests intermediate reasoning steps before a final answer. It can improve performance on some multi-step tasks, but a generated explanation is not proof that the answer is correct or that it faithfully reveals the model's decision process.
- Reasoning Models Reasoning models are language models trained for multi-step reasoning. They can spend additional computation on an answer, but gains depend on the model and task. Some offer optional extended thinking rather than using it for every response.
- Test-Time Compute Test-time compute is the processing used at inference, including generating an answer or searching and checking candidate answers. Increasing it can improve results, but the benefit depends on the task and how the computation is used.
- AI Hallucination An AI hallucination is generated content that is false, fabricated, or unsupported by the evidence it should reflect. A fluent answer can contain a hallucination even when it sounds confident or includes a citation.
From the conversations
Where guests building this work through reasoning, with full transcripts.
-
How DeepSeek Changed the AI Race Overnight Atindriyo Sanyal, Galileo -
Why LLMs Are Plausibility Engines, Not Truth Engines | Dan Klein, Scaled Cognition Dan Klein, Scaled Cognition -
Hallucinations Are a Data Architecture Problem | Sudhir Hasbe, Neo4j Sudhir Hasbe, Neo4j -
Agent Memory: The Last Battleground in the AI Stack | Richmond Alake, Oracle Richmond Alake, Oracle -
The Making of Gemini 2.0: DeepMind's Approach to AI Development and Deployment | Logan Kilpatrick Logan Kilpatrick, Google DeepMind -
The Critical Infrastructure Behind the AI Boom | Cisco CPO Jeetu Patel Jeetu Patel, Cisco
Go deeper
Short, answer-first explainers from AI, decoded.
Common questions
- What is chain of thought in AI?
- Chain of thought is a sequence of intermediate reasoning steps leading to an answer. In the 2022 chain-of-thought prompting study, examples containing those steps improved results on arithmetic, commonsense and symbolic reasoning benchmarks for the large language models tested.
- What is the difference between chain-of-thought prompting and a reasoning model?
- Chain-of-thought prompting asks for intermediate steps or provides examples containing them. Reasoning models are trained to use additional computation when producing an answer, but extended thinking is not always the default: some models offer a choice of faster responses or extended thinking.
- Does reasoning make AI more accurate?
- It can, but the benefit depends on the model, task and method. Research on test-time compute found that prompt difficulty affected which approach worked best. More computation also costs time and resources. Test answer quality, cost and latency on your workload rather than assuming that a longer explanation is more accurate.
Keep reasoning with us
New conversations every week
Chain of Thought is where builders reason through what's changing in AI, hosted by Conor Bronsdon. New conversations every week.