Reasoning Tokens

When you ask a reasoning model a hard question, it first writes out working, such as trying approaches, checking steps and correcting itself, and only then writes the answer you see. Those working tokens are the reasoning tokens. A one-line answer can sit on top of thousands of them. Providers handle them differently: OpenAI does not return them through its API, while Anthropic and Google can return a summary of the model’s thinking (OpenAI; Anthropic; Gemini API).

Hidden or not, you pay for them. All three providers bill thinking as output tokens, and the thinking counts toward output limits such as max tokens, so a low limit can cut a response off before any visible answer appears (OpenAI). They also add latency, since every token is generated before the answer starts. In episode 47 at 6:33, Google DeepMind’s Paige Bailey attributes Gemini 3 Pro’s slower answers partly to “all of the the thinking tokens” and its larger model size. Separately, at 9:21 she agrees that a lighter setting is cheaper “because you aren’t using all of the intermediate thinking tokens.” Galileo sponsored episode 47 (sponsor read at 49:58).

Most APIs let you control the amount, with a token budget, an effort level or a thinking level. More thinking tends to help multi-step problems and is wasted on simple lookups, so test settings per task. Reasoning tokens are a form of test-time compute. As with chain-of-thought prompting, a thinking summary is useful for debugging, but it is not a guaranteed faithful record of how the model reached its answer.