In detail
Agent or workflow?
Not every system with an LLM and tools is an agent in the strict sense. Anthropic’s Building effective agents draws the line by who decides the next step. In a workflow, models and tools follow paths written in code. In an agent, the model directs its own process and tool use and controls how it gets the task done. Both count as agentic systems. The same post recommends workflows for well-defined tasks, because they are more predictable and consistent, and agents where the task needs flexibility and model-driven decisions. The four parts below apply to both; what changes is how much of the routing you hand to the model. Agentic workflow and agentic vs. non-agentic AI cover the distinction.
1. Orchestration, not one model
A demo often uses a single model. A production agent system often runs a combination: small ones, large ones, some fine-tuned, some open, some proprietary, each picked for what it is good at. The job stops being “prompt the model” and becomes “coordinate the system.”
2. Memory and context
The agent needs access to your data, the memory it builds up, and the right information at the right moment. Without it, you have a clever autocomplete. With it, you have something that can work on your problem. Agent memory covers what an agent keeps between steps and sessions.
3. Tools it can call
This is where it crosses from talking to doing. The agent reaches real tools and APIs, takes actions, and operates software instead of just describing what to do. Mechanically, that is tool use: the model emits a structured call and your code runs it.
4. Evals and guardrails
The part that gets skipped. Aishwarya Srinivasan’s rule is to quantify every step, so when something breaks you can see where and why. Add the guardrails and the human-in-the-loop checkpoints, and you have a system you can trust in production instead of one you cross your fingers on.
Update, October 3, 2026: In episode 75, Kapil Chhabra describes WisdomAI’s analytics workflow: decomposing a question, generating and executing queries across data sources, verifying the returned information, repairing errors and recomposing the response. The verification and repair steps provide a concrete example of checks around the model’s output. Episode 75 was sponsored by WisdomAI (sponsor thanks at 4:13).
Why it matters
When someone says “we built an agent,” ask what system they built around the model. The model is the easy part now. The orchestration, the memory, the tools, and the checks are where it works or falls apart.
Hear it from the guest
“It is not a model that you are surfacing to your user. It is a software. It is a system that you're surfacing to your user”
“You're going to have different combination of smaller models, mid sized model, larger models. Some of them are going to be fine tuned. Some of them are going to be distilled. Some of them are going to be proprietary. Some of them are going to be open source.”
“If there is an error, it needs to self-rectify that error.”
“You're seeing a lot of definitions flying around by … all sorts of founders around, like, this is my definition. This is my definition. But at the end of the day, it's basically task completion.”
“The whole point of agents is that it does stuff rather than just … spew out tokens.”
Quotes lightly edited to remove filler words.
From the conversation
This explainer is drawn from these episodes — each carries its full transcript.
Concepts in this explainer
Agentic WorkflowLarge Language ModelAgentic AIAI AgentAgent LoopAI AssistantChatbotCoding Agent