Skip to content

AI Glossary · Concepts

What is Test-Time Compute?

Test-Time Compute

Test-time compute is the processing used at inference, including generating an answer or searching and checking candidate answers. Increasing it can improve results, but the benefit depends on the task and how the computation is used.

Also known as: inference-time compute, test time compute, inference-time scaling

· Updated · Chain of Thought

Model Architecture

  • 1 explainer
  • 2 episodes
  • 1 source

Test-time compute is the work a model or inference system uses to produce an answer, including generating tokens and evaluating candidates. It differs from computation spent training the model.

Snell and colleagues’ 2024 study examined verifier-guided search and adaptive response strategies. The most effective approach depended on prompt difficulty, so more computation alone does not promise a better answer.

Reasoning models are one way to use inference computation. Chain-of-thought prompting can elicit intermediate steps; it does not itself set an inference budget. Monte Carlo tree search is an older example from game playing, where running more simulations at decision time can improve the chosen move. Compare answer quality, elapsed time and cost for your workload before choosing a strategy. Episode 11 discusses DeepSeek, and the AI reasoning overview places these terms alongside the research.

Sources

Go deeper

  • Lightman et al.: Let's Verify Step by Step paper

    Inspect how trained reward models rank multiple candidate math solutions in best-of-N search. Separate the training feedback from the inference-time selection step and check the study's task limits.

From the conversation