How does an AI agent decide which tool to use?

In detail

Tools are described, not hard-coded

An agent doesn’t “know” your APIs. It’s handed a list of tools, each with a name and a plain-language description of what it does and when it’s the right choice. The Model Context Protocol’s tools specification is a concrete example: each tool has a name that is unique within its server, an inputSchema (a JSON Schema for the arguments it expects) and, optionally, a human-readable description. The protocol leaves the description optional, but it is the clearest guidance the model gets about when to use the tool, so write one. At each step the model reads the current task against those descriptions and decides which tool fits — or that no tool is needed and it can answer directly.

Selecting and calling

Once it picks a tool, the model generates the call: the search string, the JSON payload, the function arguments. It runs the tool, reads what comes back, and loops — using that result to decide the next action. Protocols like MCP standardize how those tools are exposed, so one agent can reach many systems through a consistent interface instead of a bespoke integration each time.

Vercel CTO Malte Ubl described that loop in episode 39 at 21:47: you give the model tools and a goal, tell it it can make “up to 10 turns using a tool,” and let it work out the sequence. It calls a tool, reads the response, and decides whether that helped or whether to call the tool again with a different query.

In a product the tool list can get long. Genspark co-founder Wen Sang said in episode 73 at 6:23, published in September 2026, that Genspark sits on top of more than 70 models, assigning planning, coding and media work to the models that are good at each. On top of that it built more than 150 in-house tools “so that models could call on these tools under the right context,” (6:23) and it pays for premium data sets such as PitchBook and Crunchbase to ground the output. Those figures describe Genspark’s own stack as he presented it. Episode 73 names Walrus Memory, Svix and Inngest as the Season 4 presenting sponsors; the reads are at 15:45, 25:40 and 41:10.

Why tool selection is where agents break

Tool choice is one of the most common failure points. If two tools have overlapping or fuzzy descriptions, the model can pick the wrong one; even with the right tool, it can pass bad arguments. Either way, the whole task derails from there. In episode 5 at 39:00, Chip Huyen named both failures as things to evaluate: the model tries to call an invalid tool, or uses the correct tool “but not with the right parameter.”

This is why “tool selection quality” is a metric worth measuring on its own, and why writing precise tool descriptions is one of the most direct ways to improve it. Anthropic’s tool-use documentation calls extremely detailed descriptions “by far the most important factor in tool performance”, and its engineering guide to writing tools warns that too many or overlapping tools can distract an agent, recommending namespaced names that mark the boundaries between them. For comparing models rather than your own tool set, the Berkeley Function Calling Leaderboard evaluates how accurately models call functions.

Marking tools that change things

Some tools only read; others edit, delete or send. In episode 48 at 29:25, Block’s Angie Jones described annotating tools as destructive so the agent knows which ones can change things, plus a setting in goose that lets the agent run non-destructive tools freely but ask the person’s permission before a destructive one. MCP has a field for such tool annotations, but its specification says clients must treat them as untrusted unless they come from trusted servers, so the permission check belongs in the client, not in the server’s self-description.

Disclosure: Episode 39 was sponsored by Galileo while Conor worked there. Episode 48 was sponsored by Galileo, the Season 3 presenting sponsor; it thanks Galileo at 49:33.