How modern AI works, in plain English

1.1 What an LLM actually is

1. How do large language models actually work?

   A large language model is a neural network trained to predict the next token, a word or a piece of a word, from the text that came before it. Pre-training on a huge amount of text teaches it the patterns of language and a lot about the world. Post-training on example conversations and feedback from people or other models then turns that text predictor into an assistant that follows instructions. Thomson Reuters CTO Joel Hron told Chain of Thought its legal model starts from an open-source model, then goes through continuous pre-training on its own content, fine-tuning, and agentic reinforcement learning.

1.2 Tokens, context windows and cost

1. What is a token in AI, and why does it decide what AI costs?

   A token is the unit a language model reads and writes: a word, part of a word, or a piece of punctuation. In English a token averages about four characters, or three-quarters of a word. Providers bill per token, with separate prices for what goes in and what comes out, so the cost of an AI feature depends on how many tokens each task uses, not just on the price per token. Cisco's Jeetu Patel told Chain of Thought that price per token might become a vanity metric, because long-running agents can use more tokens even as the per-token price falls.

2. What is a context window in AI, and why does it fill up?

   A context window is the token budget a language model can use for a request and its response. Instructions, retained conversation, documents, tool results and tool definitions share that budget. Keeping more history makes it grow; applications can remove or summarize older material. More input can increase cost and make useful information harder to find. WisdomAI's Kapil Chhabra told Chain of Thought that the trick is to put in just the right amount of context for each prompt, because too much makes the model guess and too little leaves gaps it fills by guessing.

1.3 Why AI makes things up

1. Why do LLMs hallucinate?

   Language models can produce plausible text without establishing that it is true. Missing or rare facts, incomplete context and training or evaluation that rewards guessing can all contribute to confident but false answers.

2. Are AI hallucinations always bad?

   No. A hallucination is the model generating something not grounded in fact, and whether that is bad depends entirely on the use. It is a feature for creative work and dangerous for anything factual, with the worst case being an answer that looks right but is wrong in context. NVIDIA's Vivienne Zhang told Chain of Thought that when a chip designer asking about CTL means Compute Trace Library, an answer using the term's generally accepted meaning is as unhelpful as a completely wrong one.

3. Are AI hallucinations a data problem or a model problem?

   Largely a data problem, especially in the enterprise. A language model predicts plausible text; when it lacks the right grounding it fills the gap with something that sounds right, which we call a hallucination. Much of that comes from the data layer — missing context, stale or contradictory sources, poor retrieval, no single source of truth. You can't fully train hallucination out of the model, but you can starve it: ground answers in trusted, current data and the model has less reason to invent. The model generates; the data decides whether it has the truth to generate from. Neo4j's Sudhir Hasbe told Chain of Thought that customers using only vector-search RAG see around 60% to 65% accuracy, while some combining graph and vector search are well above 90%.

1.4 The model landscape

1. When should you use a reasoning model instead of a standard LLM?

   Use extra reasoning when tests on representative tasks show a quality improvement worth the added time and cost. Compare models or thinking modes on the same workload, verify the results, and include difficult cases rather than assuming a longer explanation is better.

2. When should you use a small language model instead of a frontier model in production?

   Default to a frontier model while you're figuring out what 'good' looks like — its broad capability lets you prototype fast without fighting the model. Move a task to a smaller model once it's well-scoped and high-volume, because that's where cost and latency dominate and a small model tuned to one job can match frontier quality at a fraction of the price. The frontier stays the right call for open-ended reasoning, low-volume work, and requirements that are still moving. The decision isn't 'which model is smartest' — it's which model is the cheapest, fastest way to clear the quality bar your specific task needs. On Chain of Thought, Intercom's Fergal Reid said a post-trained 14B Qwen model replaced GPT-4.1-class models on a summarization task that may have cost a quarter million dollars a month, saving almost all of it.

3. Should you use open source or proprietary LLMs?

   It depends on the job, and most serious teams use both: open when you need control, customization, privacy, or cost efficiency; proprietary when you need top-end quality on certain tasks or the easiest path to start. No one has won the race, so locking into one provider is the mistake. Fireworks AI's Aishwarya Srinivasan told Chain of Thought that DeepSeek V3 was an inflection point for the performance-to-price ratio of open-source models, while vision remained harder for available open models than proprietary ones.

4. Is an open-weight AI model the same as an open-source one?

   Not necessarily. An open-weight model publishes its trained parameters so you can download, run and fine-tune it. An open-source AI system, under the Open Source Initiative's definition, also has to provide the training code and enough information about its training data to rebuild it, under terms that allow any use. Most popular open models are open-weight, with licenses that range from Apache 2.0 to custom terms with use restrictions. Intercom's Fergal Reid told Chain of Thought it was really generous of the Qwen team to release the Qwen 3 models as open weights, given the large training expense to create them.

1.5 From chatbot to agent

1. What makes an AI agent different from an LLM?

   An LLM answers; an agent does. The difference is the system built around the model, which in production means four things: multiple models orchestrated together, memory and context, tools it can call, and a layer of checks running the whole time. The model is just one part of the system. Fireworks AI's Aishwarya Srinivasan told Chain of Thought that Notion's head of AI engineering described fine-tuning models on Fireworks for latency-sensitive parts of Notion AI, which uses multiple models.

2. What's the difference between agentic and non-agentic AI?

   Non-agentic AI runs a fixed path: you give it an input, it returns an output, done — a chatbot answering a question, a model classifying a document. Agentic AI runs a loop: it sets a sub-goal, takes an action, observes the result, and decides what to do next, repeating until the task is done. The line is autonomy over the steps. Non-agentic systems follow a path you defined; agentic systems decide the path themselves, which is more capable and far harder to predict. Scaled Cognition's Dan Klein told Chain of Thought that a RAG question-answering loop isn't agentic to him; agentic means goals at play and actions that have consequences.

3. What is multimodal AI?

   Multimodal AI is a model that takes in and reasons across more than one kind of data — text, images, audio, video — in a single system. Instead of a separate model for each, one model can read a chart and answer questions about it, transcribe speech and act on it, or describe a video, and some multimodal models can now generate images as well. A hard part isn't handling each modality; it's alignment — getting the model to connect what it sees, hears, and reads into one coherent understanding. Google DeepMind's Logan Kilpatrick told Chain of Thought that a successful domain-specific computer vision project once took roughly nine to twelve months; models now handle segmentation and object detection.

Also at this level

1. Is using AI bad for the environment?

   Not at the level of your own use, and many of the figures in circulation are wrong by orders of magnitude. The aggregate question is the one that bites: how much power the data-center buildout draws and where that power comes from. Utilities and siting decisions move that far more than how often any one person opens a chatbot. On Chain of Thought, Andy Masley said sending a ChatGPT prompt probably emits as much as running a space heater for about half a second or driving a car about four feet.

2. How much electricity do AI data centers use?

   Data centers of every kind used about 415 terawatt-hours (TWh) of electricity in 2024, roughly 1.5% of the world's total, and the International Energy Agency projects about 945 TWh by 2030, with AI servers driving much of the growth. The strain is local: US data centers used about 4.4% of the country's electricity in 2023 and could reach 6.7% to 12% by 2028. Cisco's Jeetu Patel told Chain of Thought that data centers are now being built where power availability is highest, instead of power being brought to them.

2.1 Brief it like a colleague

1. How do you give an AI assistant a useful brief?

   Specify the outcome, supply the evidence it needs, define the output, and state what it may do. Include a checkable example and an instruction for missing information. A useful brief makes success and the permission boundary clear enough that someone else can review the result. LlamaIndex's Jerry Liu told Chain of Thought that a request to prepare for a customer call isn't enough; the agent needs the previous 5 to 10 transcripts and recent email threads.

2.2 Give it the right context

1. What is context in an AI agent?

   Context is the information and capabilities supplied around a model so it can carry out a task. Jerry Liu describes documents, web information, structured data, connected tools, and systems of record as parts of that surrounding layer. These categories overlap; they are a way to inspect an agent’s inputs and access, not a universal architecture. On Chain of Thought, Slack CPO Jaime DeLanghe said an agent working inside a Slack channel has access to that channel, based on the same permission model people already use in Slack.

2. What is RAG, and why do AI systems use it?

   RAG, retrieval-augmented generation, is a pattern where the system fetches relevant documents at query time and hands them to the model along with the question, so the answer is grounded in real sources instead of the model's memory. It exists to fix two problems with a bare language model: it doesn't know your private or current data, and it makes things up when it doesn't know. RAG gives the model the right context to read before it answers.

2.3 Check its work

1. How do you check whether an AI answer is supported?

   Break the answer into claims and compare each material claim with an authoritative source. Check dates, scope and qualifications, then verify calculations or recorded actions independently. A citation proves that a source was named; it does not prove that the source supports the answer. On Chain of Thought, MongoDB's Mikiko Chandrasekhar described how a few summer reading lists from reputable sites apparently included books that did not exist, with criticism centered on the missing editorial review.

2.4 Agents you can use today

1. Can an AI agent actually run your workday?

   Postman's Sterling Chin says an agent he built runs 90% of his day, and the two things that made it work are unglamorous: scoping, and treating it like a junior hire. Worth knowing before you copy it: the 90% is his own estimate, and the productivity number behind it comes from a report the agent wrote about itself. Chin told Chain of Thought his agent couldn't know which of thousands of JIRA tickets mattered until he shared links to his own Confluence space and Kanban board.

2. What can MCP actually do?

   MCP lets an AI agent connect to your real tools and chain them together, which a plain chatbot cannot do. The value is not in any single connection but in wiring several systems into one workflow. On Chain of Thought, Block's Angie Jones described the company's Goose agent, invoked in Slack and connected to the GitHub MCP, turning a reported bug into a pull request in five minutes.

3. How much autonomy should you give an AI agent?

   As much as the risk of the task allows, and no more. There is no single right answer; you climb the ladder one step at a time and decide at each step whether a human still needs to sign off. Neat's Tormod Ree told Chain of Thought that teams should be more aggressive letting AI coding loose on less dangerous adjacent work and more careful with platform code or DevOps environments.

2.5 What’s safe to paste

1. How do you decide what information an AI assistant may receive?

   Check the task’s data needs, your authorization, and the exact product or provider route’s handling terms. Send only the necessary permitted material. Keep credentials out of prompts, and enforce access in tools and downstream systems; neither a paid plan nor a model name establishes permission to share private information. Ledger CTO Charles Guillemet told Chain of Thought he wouldn't give his crypto private keys to an agent, since it might act against his intent or be prompt-injected into revealing them.

Also at this level

1. Does AI replace human translators?

   Not yet, and Smartling's Olga Beregovaya is specific about where the gap sits. Models are trained mostly on English phenomena, so quality falls away on less-represented languages while the output still reads fluent, and the failure is factual and cultural rather than grammatical. Her framing is that AI does the heavy lifting and a human covers the delta. Beregovaya told Chain of Thought that a translator can deliver 2,000 words a day on average, and that a well-trained, well-prompted model brings lower costs, higher productivity, and much more predictable quality.

3.1 Where to start

1. Is the AI agent bubble real?

   There's a gap between the hype and what ships. Demos of autonomous agents are everywhere; reliable agents running unattended in production are rare, and guests describe most enterprise AI work as never getting past proof of concept. Expectations assumed capability that the engineering hasn't delivered yet. The bubble is in those expectations and the timeline. Lyzr AI CEO Siva Surendira told Chain of Thought that 95% of all AI work done by enterprises remains at the proof-of-concept stage and doesn't move into production.

3.2 Build, buy or rent

1. Should you use prompting, RAG, or fine-tuning to customize an AI model?

   Start at the cheapest rung and only climb when you must. Prompting shapes behavior with no infrastructure; RAG grounds the model in your data so answers stay current and citable; fine-tuning changes the model's weights to bake in a style, format, or skill. Most teams need prompting plus RAG, and reach for fine-tuning last — for how the model should behave, not what it should know. Intercom's Fergal Reid told Chain of Thought that a production prompt can accumulate something like 50 special cases; post-training can absorb them and shorten the prompt.

2. Can switching to an open model actually cut your AI costs?

   On the narrow, high-volume tasks, yes. Intercom was spending roughly $250,000 a month, by its Chief AI Officer's own estimate, running one summarization job through a closed-model API and replaced it with a fine-tuned 14-billion-parameter Qwen model. The decision is made per task rather than per platform: the hardest prompt in the same product still runs on a frontier model. Galileo CTO Atindriyo Sanyal told Chain of Thought that self-hosting open models carries a larger upfront cost for development and configuration, but at scale costs significantly less than an LLM provider's API.

3. Why do some enterprises need to run AI on-premise?

   Because for regulated industries, the data can't leave the building. Sending prompts and documents to an outside AI provider means your sensitive data — patient records, financial data, regulated IP — crosses a boundary your compliance team can't allow. Running the models and the observability stack on-premise, behind your own firewall, keeps the data, the audit trail, and the control inside your perimeter. It costs more and is harder to operate, which is why it's a requirement for the regulated, not a default for everyone. It is also no longer limited to open models, since some providers now ship frontier models for air-gapped data centers. Poolside CTO Eiso Kant told Chain of Thought that his team focuses on bringing models safely behind customers' firewalls, where the data is, instead of sending customer data out.

3.3 Pilot to production

1. Why do most enterprise AI projects fail to show ROI?

   Most stall before they ever reach the scale where returns show up. The pilot demos well, then the project hits the costs nobody budgeted: evaluation, integration with messy real systems, data cleanup, governance sign-off, and the ongoing expense of running and monitoring the thing. Add a vague success metric — 'improve productivity' with no baseline — and you get projects that consume budget without producing a number anyone can point to. The failure is usually operational and organizational, not the model. On Chain of Thought, Accenture's Jay Subrahmonia cited stats that fewer than 10 percent of AI proofs of concept reach production, partly because the governance, testing, and monitoring aren't in place.

2. How do you measure whether AI is actually paying off?

   Measure time back, not usage. Count the human work an agent actually removed: work that took a person five hours a week and now takes an agent five minutes pays off; using AI to redo a font you could have fixed in one click does not. Jiaona Zhang calls the second case 'token maxing.' Get visibility into spend against the outcome you're driving — revenue or time-allocation efficiency — and articulate that outcome before you count. Slack CPO Jaime DeLanghe told Chain of Thought not to assume faster coding agents fix a cumbersome development process when their code still has to clear 20 checks from 20 different teams.

3.4 Evals for leaders

1. What are AI evals, and what should leaders know about them?

   An AI eval is a repeatable test of whether an AI system does its job well enough: a set of representative inputs, a definition of a good answer, and a way to score the output, rerun as the prompt, model or data changes. For leaders, evals are how you find out a feature fails before customers do, and the people who know the business have to help define what good means. Independent AI consultant Hamel Husain told Chain of Thought that outsourcing evaluations to developers is one of the biggest failure modes he sees, because developers usually aren't the domain experts.

2. What's the difference between AI observability, evaluation, and benchmarking?

   Benchmarking compares models against a fixed dataset before you pick one — it answers 'which model is better in general.' Evaluation measures whether your system does the right thing on your task and your data — 'is this good enough to ship.' Observability is what you run in production — tracing live behavior to see what actually happened when something broke. They answer different questions at different stages, and teams get into trouble by using one where they need another. Intercom's Fergal Reid told Chain of Thought that optimizing for a benchmark can produce generalization failure, and that his team always runs a production test of a well-prompted, untrained model before post-training one.

3.5 Guardrails, risk and regulation

1. What's the difference between human in the loop and human on the loop?

   Human in the loop means a person takes part in each decision, usually by approving actions before the AI carries them out. Human on the loop means the AI acts on its own while a person monitors it and can step in, override or stop it. Which one fits depends on what a mistake costs and how much evidence you have that the system gets it right, and the safer path is to move from in to on one task at a time. Cisco's Jeetu Patel told Chain of Thought that teams should approve every agent action until they have seen enough to be statistically confident, and only then consider taking a human out of the loop.

2. How do you govern AI agents in an enterprise?

   You govern agents the way you govern any system that takes consequential action: know what they are, control what they can do, and keep a record of what they did. In practice that means an inventory of every agent in production, scoped permissions and approval gates on high-stakes actions, audit trails of decisions and tool calls, and a named owner accountable for each one. The reason it matters now is trust — agents have to earn trust before they act on real systems, and governance is how you earn the right to deploy them. Redpanda CTO Tyler Akidau told Chain of Thought that agent permissions should expire: an agent given billing-database access for a 2 p.m. job shouldn't still have it an hour later, maybe even a minute later.

3. How do enterprises let employees use AI agents safely?

   Four guardrails: an allowed list of approved connectors, identity-based authentication, flags on destructive actions, and a human in the loop for anything risky. That is how Block runs AI agents across 12,000 employees at a company handling Square and Cash App. Block's Angie Jones told Chain of Thought that every MCP server on that allowed list, even ones her own team built, goes through a security review first, and most were built in-house.

4. What does it actually take to ship AI in a regulated industry?

   Provenance on every component, not a single passing benchmark. Corti CEO Andreas Cleve's requirement is that you can explain what each part of the pipeline did, why it changed, and what went into it. He also argues that public leaderboards are close to useless as evidence, because the models topping medical benchmarks are the ones nobody ships. Cleve told Chain of Thought that doctors must review Corti's output, giving the company a gold-standard label for whether each result is complete and concise enough to use.

3.6 Teams and skills

1. Is AI taking software engineering jobs?

   The hiring data discussed with Greenhouse CEO Daniel Chait points to the matching layer breaking, not the jobs vanishing. Figures cited in the episode put applications up 239% since ChatGPT while 75% fewer reached the hire stage, and software engineers are the heaviest users of the automation driving that. Much of it is a signal problem, and each side makes it worse by responding rationally to the other. Separate research finds employment for young workers in AI-exposed occupations lagging their peers, a pattern its authors do not yet attribute to AI. Chait told Chain of Thought that an individual recruiter may be looking at four or even five times as many applications as just a couple of years ago.

2. Who reviews the code when AI writes most of it?

   A human still owns the merge, but the review cannot stay a line-by-line read of the diff. Generation stopped being the constraint the moment background agents could open pull requests unattended, and the teams keeping up moved the check that decides into the test harness. AMD's Anush Elangovan told Chain of Thought that testing is the code review: the test harness sets hard requirements agents must meet, giving a 50-microsecond dispatch time in his GPU simulator as an example.

Also at this level

1. Do you have to centralize your data before AI agents can use it?

   No, and Starburst's Jitender Aswani argues that moving everything into one place stops scaling as sources multiply. A traditional enterprise runs 52 to 200 data sources and the count keeps climbing, so a plan that ends with everything in one place keeps chasing new sources. Federation puts a query layer over the data where it already lives, which he thinks is the model that keeps working as the number of sources grows. Aswani told Chain of Thought that when a SaaS vendor adds a new column, teams must retool their ETL and can spend days debugging why pipelines are failing.

2. Can AI modernize legacy code and old applications?

   It can do a lot of the work, but not unsupervised. AI is good at the slow parts of modernization — reading undocumented code, explaining what a function does, translating between languages, and drafting migrations. Where it can fail is the part that matters most: it may not know the business logic and edge cases the old system encodes, so it can confidently rewrite something subtly wrong. The pattern that works is AI as an accelerator with engineers verifying, plus tests that prove the new code behaves like the old. MongoDB's Rachelle Palmer told Chain of Thought she believes an 80/20 rule applies: LLMs can assist with 80% of modernization work, but the remaining 20% is purely manual effort.

4.1 Retrieval and RAG

1. What is agentic RAG, and how is it different from regular RAG?

   Traditional RAG runs one fixed retrieve-then-generate step: fetch documents that match the query, stuff them in the prompt, answer. Agentic RAG puts an agent in charge of retrieval — it decides whether to search, reformulates the query, pulls from multiple sources, checks whether what it got is good enough, and retrieves again if it isn't. The difference is a static pipeline versus a control loop. Databricks' Craig Wiley told Chain of Thought that in 2024 teams found a prompt plus some docs isn't a system to bet a company's reputation on, and production-grade systems are often compound or agentic.

2. Vector database or knowledge graph — which should you use for AI retrieval?

   Use a vector database when relevance is about meaning — finding passages similar to a question across unstructured text. Use a knowledge graph when the answer depends on explicit relationships and facts — who connects to what, and how. They complement each other: vectors find the right neighborhood, a graph enforces the right facts, and some teams pair them to cut hallucination. On Chain of Thought, Neo4j's Sudhir Hasbe said customers starting with pure vector search saw accuracy around 60 to 65%, while some combining graphs and vectors are well above 90% accurate.

3. How do you prepare PDFs for a RAG system?

   Extract the document’s text and structure, check that tables and reading order survive, and keep source locations before splitting it into retrievable chunks. Test the extracted representation against the original pages; a search pipeline cannot recover a qualification or table relationship that preparation discarded. LlamaIndex CEO Jerry Liu told Chain of Thought that frontier models get roughly 80% accuracy across a document corpus; about 20% of documents yield a hallucinated number or a misrepresented table.

4. What is a reranker in a RAG system?

   A reranker takes the candidates returned by search and scores them again for relevance to the question, so the answering model receives a better ordered shortlist. It can improve which retrieved passages reach the prompt, but it cannot select a relevant document that the first search never found. Intercom's Fergal Reid told Chain of Thought that custom retrieval and re-ranker models drive a lot of Fin's accuracy, and the re-ranker outperformed the best Cohere re-ranker at the time, which Intercom had used.

5. How do you evaluate a RAG system?

   Evaluate source coverage, retrieval and generation separately. Check whether the needed information exists, whether the system retrieves it, and whether the answer is faithful to that evidence and answers the question; one end-to-end score can hide which stage failed.

4.2 Context and memory engineering

1. What is the difference between prompt, context, and memory engineering?

   They are three different jobs, and they happen in order. Prompt engineering is how you word the request. Context engineering is what you put in front of the model for a single task. Memory engineering is what the system moves out to an external store and pulls back later. Oracle's Richmond Alake told Chain of Thought that summarizing a conversation back into the context window is context engineering, while moving the uncompressed information to a database or files for just-in-time retrieval is memory engineering.

2. What are the types of AI agent memory?

   Richmond Alake describes four types: working memory holds the current task, semantic memory stores facts, episodic memory records past events, and procedural memory stores how to do a task. This human-memory analogy describes their uses; it does not determine which storage system to buy or which remembered fact should win a conflict.

3. How do you make an agent's memory portable across apps?

   Keep approved memory records in an exportable form, define how another application reads and updates them, and test retrieval after changing apps or models. Include provenance, access scope, corrections, and deletion handling in the transfer contract. A successful import proves that records moved; it does not prove equivalent recall or ongoing synchronization.

4. What is the difference between a semantic layer and a context layer for AI?

   A semantic layer defines business metrics and how to calculate them consistently. A context layer is a broader, less standardized term for the information an AI system needs to use those definitions correctly, such as the user’s intent, calendar conventions, worked queries and task instructions. WisdomAI's Kapil Chhabra told Chain of Thought that a semantic layer is necessary but not sufficient for chat agents, which need context such as a fiscal year ending January 31st rather than December 31st.

5. What is data lineage, and why does an AI system need it?

   Data lineage records where data came from, how it changed and which downstream datasets or systems used it. For AI, connect that history to model, dataset and pipeline versions so you can investigate a bad result, identify affected outputs and decide what needs to be rebuilt or retested. IBM's Maryam Ashoori told Chain of Thought about an insurance company that couldn't use model customization without lineage and governance, because it needed to know which model version was trained on which data version.

4.3 Tools, MCP and agent architecture

1. How does an AI agent decide which tool to use?

   The agent is given a set of tools, each with a name and a description of what it does and when to use it. At each step the model reads the task and those descriptions and picks a tool, then generates the arguments to call it — a search query, an API payload, a database lookup. It runs the tool, reads the result, and decides the next move. The quality of that choice rides heavily on the tool descriptions: vague descriptions invite wrong tool calls, which is one of the most common ways agents fail. On Chain of Thought, Vercel CTO Malte Ubl described giving a model descriptions of what its tools are good for, a goal, and up to 10 tool-using turns to work out the steps itself.

2. How do AI coding agents like Claude Code work?

   An AI coding agent is a language model running in a loop with tools to read and search files, edit them and run commands such as the test suite. Its available tools and permissions depend on the harness; some can also open pull requests. The model picks a tool, reads what comes back, and decides the next step until the task is done, a limit is reached or it needs a person. Anthropic describes the Claude Code loop as gather context, take action and verify results. Vercel's Malte Ubl told Chain of Thought that this simple architecture, an LLM with a few plain tools, is why Claude Code works as well as it does.

3. How does MCP (Model Context Protocol) work?

   An AI application, called the host, opens one MCP client connection for each MCP server it uses. Each server offers tools to call, resources to read and prompts to reuse, and the two sides exchange JSON-RPC 2.0 messages over standard input and output for a local server or Streamable HTTP for a remote one. The model never connects to your systems directly; the host decides what it sees and should ask a person before risky calls. Block's Angie Jones told Chain of Thought that Block rebuilt its Goose agent as an MCP client after Anthropic showed the company a draft of the protocol.

4. Should you use MCP or build a custom integration to connect AI to your tools?

   Use MCP when a tool or data source will be reused across multiple models, apps, or teams — you integrate it once and everything speaks to it. Build a custom integration when you need a tight, high-control connection to a single system and the standard overhead isn't worth it. For most organizations scaling agents, MCP wins because it kills the N-models-times-M-tools integration explosion; custom is the exception for bespoke, performance-critical paths. Block's Angie Jones told Chain of Thought that with data stores connected through MCP, employees can query them in natural language from their preferred MCP-compliant agent instead of waiting maybe a week for a report.

5. Do you still need an AI agent framework?

   Choose a framework when its orchestration and state-management features solve problems your agent has. In his Chain of Thought interview, LlamaIndex co-founder Jerry Liu argues that stable abstractions leave more room to differentiate through production context. That is a reason to evaluate what a framework adds, not a rule that every team should abandon one. Liu also points out that frameworks caught on partly because models then had 4,000-token context windows, too small to hold a company's knowledge base.

6. Which AI agent framework should you use — LangGraph, CrewAI, or AutoGen?

   LangGraph emphasizes explicit state and orchestration. CrewAI combines role-based teams with structured Flows. AutoGen supports multi-agent applications, but its README now says Microsoft has archived it and no longer maintains it, and directs new users to its successor, Microsoft Agent Framework. Choose around the workflow you need to control and the project’s current support status. CrewAI founder João Moura told Chain of Thought he hears the same thing across the market: no one wants lock-in, and teams want to change LLMs, frameworks, or memory at any point.

4.4 Evaluation and observability in practice

1. How do you test an AI system when the output isn't deterministic?

   Keep exact tests for deterministic contracts such as permissions, schemas and recorded actions. For flexible language, test required properties on representative cases and use a rubric calibrated against people. Repeat important cases and keep a held-out set so a release check catches failures beyond the examples used to tune the system. AI consultant Hamel Husain told Chain of Thought that to trust an LLM judge you have to compare it to human labels, and he suggests reading at least 100 traces as a starting heuristic.

2. How do you use synthetic data to test an AI system?

   Use synthetic data to draft test cases from trusted source material and known failure patterns, then have qualified reviewers check the questions, expected answers and coverage. Keep real-world cases and an independent holdout alongside generated tests so the generator does not become its own answer key. Snorkel CEO Alex Ratner told Chain of Thought that review and quality control take almost 40% to 50% of the time spent creating a data point or environment to Snorkel's quality standard.

3. What is LLM-as-a-judge, and when can you trust it?

   LLM-as-a-judge uses one language model to score the output of another against a rubric — is this answer relevant, grounded, complete, safe. It scales evaluation past what humans can read by hand. You can trust it when you've calibrated it against human judgments on your own data, given it a concrete rubric, and kept a person in the loop for the high-stakes calls. Used blind, it inherits the same biases as the model doing the grading. Scaled Cognition CTO Dan Klein told Chain of Thought that a model and its checker fail on exactly the same hard cases, so an 80%-right model checking an 80%-right model gets to about 82%.

4. Should you evaluate AI with an LLM-as-a-judge or with human review?

   Use an LLM-as-a-judge for scale and speed — scoring thousands of outputs continuously, catching regressions, and ranking A vs B. Use human evaluation for ground truth — defining what 'good' means, judging nuance and high-stakes cases, and calibrating the judge. Use them together: humans set and audit the standard, the LLM judge applies it at volume. Galileo CEO Vikram Chatterji told Chain of Thought that, taking GPT-5 as an example, an LLM judge is around 70% accurate, wrong three out of 10 times.

5. How do you evaluate an AI agent?

   Check the final outcome in the system of record, including authorization and unwanted side effects. Use step, turn and session checks to diagnose failures; a convincing reply or a correct tool call does not prove the task succeeded safely.

6. What is AI observability, and why do you need it in production?

   AI observability is instrumenting an AI system so you can see what it actually did on each request — the retrieved context, the tool calls, any reasoning the model exposes, the final output — instead of just whether it succeeded or failed. You need it because AI systems are non-deterministic: the same input can behave differently, failures are silent, and a confident wrong answer looks identical to a right one. Without traces of the real behavior, you can't debug, you can't catch drift, and you can't tell a working system from one that's breaking. Redpanda CTO Tyler Akidau told Chain of Thought that since agents can't be debugged like classic software, the only real recourse is to record every prompt, input, tool request, tool response, and output.

7. What should you measure on an AI agent besides accuracy?

   Accuracy tells you whether the final answer was right, but it hides how the agent got there. The metrics that predict reliability watch the process: did it pick the right tool, call it correctly, and recover when something failed; how many steps and how much it cost to finish; whether it stayed on the user's intent across a long conversation; and how often it needed a human to step in. An agent can be accurate in a demo and unreliable in production because none of those were measured. Scaled Cognition's Dan Klein told Chain of Thought that a system scoring 80% or 90% can't really ship, while one that gets the same 70% of scenarios right every single time is hugely valuable.

4.5 Customizing models

1. How do you prepare data for fine-tuning an LLM?

   Build reviewed examples that match the inputs, conversations and outputs your model will face in use. Cover both routine cases and difficult boundaries, keep evaluation examples separate, and improve the dataset by reading the errors your trained model makes. Liquid AI's Maxime Labonne told Chain of Thought that post-training means spending most of your time on the data, including reading model responses and training samples, which he said people don't do enough.

2. What is model distillation in AI?

   Model distillation trains a student model to reproduce useful behavior from a teacher model, often transferring a narrow capability into a smaller model. The training signal may be the teacher’s answers or its predicted probabilities; the student still needs independent testing on the task it will perform. Liquid AI's Maxime Labonne told Chain of Thought that knowledge depends on parameter count: no amount of distillation will make a 1B model as smart as a 3B model.

3. How does reinforcement learning improve AI coding models?

   Reinforcement learning trains a coding model using feedback on the programs it generates, such as whether they pass tests. Repeated generation, scoring and model updates can favor successful behavior, but the improvement depends on whether the tests and reward capture the task you actually want solved. Poolside CTO Eiso Kant told Chain of Thought its models learn from code execution feedback across almost a million repositories that Poolside has fully containerized with their full test suites.

4.6 Cost, latency and safety basics

1. How do you cut the cost of running an AI agent?

   Trace cost per verified successful task, including failures, retries and human handling. Then test cheaper models, smaller context, caching and fewer tool calls against the same quality and latency requirements.

2. How do you make an AI feature respond faster?

   Measure the full path from the user’s action to a useful result, then shorten the steps that block it. Prepare stable context ahead of time, reuse eligible cached work and run independent steps together, while checking that freshness, answer quality and total cost still meet the task’s requirements. Superhuman's Loïc Houssier told Chain of Thought that getting LLM inference under the product's 100-millisecond bar doesn't work, and that for voice drafting his team pre-loads voice and tone context, then sends just the transcription.

3. Are small language models better than large ones for production?

   Often, yes — for a specific, well-defined task. A small model that's been tuned for your job can match a frontier model's quality on that job while costing far less, running faster, and being possible to host yourself. The frontier models earn their keep on broad, open-ended reasoning. The mistake is defaulting to the biggest model for everything; the production-smart move is using the smallest model that still passes your evals for each task. Intercom's Fergal Reid told Chain of Thought that Fin has run over 10 trillion tokens in production, and shifting 30% or 50% to a lighter custom model would mean real margin and lower latency.

4. What are AI agent guardrails, and how do you set them?

   AI agent guardrails are checks and controls around requests, tool calls and responses. Some use a model to judge content; others validate arguments or enforce permissions. Set them by naming the failure you want to prevent, choosing the check and placing it before the effect it must stop. A model-based check can make mistakes, and a final-output filter cannot undo an action already taken. Keep action authorization outside the agent's control and test the paths, timing and failure behavior of the whole workflow.

Also at this level

1. How do AI voice agents work?

   Most voice agents use one of two designs. A chained pipeline turns speech into text, sends the text to a language model and turns the reply back into speech. A speech-to-speech model takes audio in and returns audio out from one model over a streaming connection. The second cuts handoffs and delay; the first is easier to inspect and control step by step. Yutori's Dhruv Batra told Chain of Thought that spoken conversation with a person carries a hundred-millisecond latency constraint, and anything slower causes unnatural pauses.

5.1 Multi-agent systems

1. Should you build a single agent or a multi-agent system?

   Start with a single agent. One agent with a clear set of tools is easier to build, debug, and trust, and it handles many tasks. Reach for multiple agents only when the work splits into distinct specialties that benefit from separate context and instructions — and accept that you're trading simplicity for new failure modes: coordination overhead, agents talking past each other, and harder debugging. Multi-agent is a way to manage complexity, not a free upgrade. Block's Angie Jones described on Chain of Thought a one-hour MIT workshop where six specialized sub-agents built a full-stack app, and the last, a QA sub-agent, said not to ship it to production.

2. What is the difference between MCP and A2A?

   MCP connects an AI application to tools and context; A2A defines how separate agents communicate and coordinate work. They can be used together: an agent may accept a task through A2A, then use MCP-connected tools to carry it out. Cisco's Giovanna Carofiglio told Chain of Thought that MCP is aimed at agent-to-tool interaction, while agent-to-agent communication, where protocols such as A2A are emerging, is the trickier problem.

3. Why do multi-agent systems fail, and how do you make them reliable?

   Multi-agent systems fail in the gaps between agents, not inside any one of them. Small per-agent errors compound: a handoff drops context, one agent's wrong output becomes another's trusted input, and a minor fault cascades into a systemic failure no single agent would have produced alone. You make them reliable by treating the system as the unit — tracing every step, validating what passes between agents, setting guardrails on autonomy, and threat-modeling how faults propagate before they reach production. Lyzr CEO Siva Surendira told Chain of Thought that a feedback and conflict-resolution loop between agents ran up $300 in LLM calls without fixing the problem before Lyzr stopped it.

5.2 Agent security and identity

1. How do you give an AI agent an identity and permissions?

   Give each agent instance a short-lived identity scoped to one task, then limit its authority to the intersection of the user's rights, the task, the agent's restrictions, and enterprise policy. Enforce those limits somewhere the agent cannot see or modify, because access to a system is not authority to do everything inside it. Cisco's Jeetu Patel told Chain of Thought that agents need action control as well as access control: an agent in your inbox can book dinner reservations but shouldn't start texting your board of directors.

2. Can AI agents be secured with software alone?

   Software controls can enforce scoped access and approved actions, but a model’s instructions alone cannot secure the systems it can reach. Ledger CTO Charles Guillemet argues for hardware-backed signing when protecting valuable assets. Match that layer to the threat model, and enforce authorization outside the model. Redpanda's Tyler Akidau told Chain of Thought that telling an agent never to delete the database works until it doesn't, so governance must run through channels the agent can't access, modify, or see.

3. What is context poisoning, and how do you stop it?

   Context poisoning is what happens when an agent's context window fills with wrong, stale, or badly shaped data and the agent reasons off it. The fix sits upstream of the model: control what reaches the window, because one ordinary query against a badly shaped API can burn tens of thousands of extra tokens on data the agent did not need. On Chain of Thought, Airbyte CEO Michel Tricot demoed a simple Gong query that, run through raw API calls, burned almost 30,000 extra tokens.

5.4 Infrastructure choices

1. Why does AI run on GPUs?

   Because training and running a neural network is mostly matrix math that splits into thousands of small calculations, and a GPU is built to run thousands of threads at once while a CPU is built to run a few threads as fast as possible. Gaming demand made that parallel hardware cheap; memory and networking now set the limits as models outgrow a single chip. Databricks' Carly Taylor told Chain of Thought that gaming drove down the cost of each matrix calculation on a GPU, and that today's data center GPUs are used primarily for generative AI.

2. Why is it so hard to move AI workloads off NVIDIA?

   Because much of the switching cost lives in the software, not the silicon. Memory and math formats are catchable on a roadmap; what is hard to match is an ecosystem where new frontier models run on day one, which is what AMD has spent the last few years working to match and what Google sidestepped by owning its stack from the compiler up. AMD's Anush Elangovan told Chain of Thought that a few years ago ROCm support meant porting each model after launch, but recent frontier models such as DeepSeek, Llama and Qwen had day-zero support.

5.5 The frontier

1. Why won't most websites get APIs for AI agents?

   Most websites will never expose agent APIs, argues Dhruv Batra of Yutori. The long tail that runs the internet, tens of thousands of school district sites, government offices, and hundreds of thousands of e-commerce pages, was built for humans and has no reason to re-architect. He says some of the resistance is socio-political, which coding agents don't solve. His model: agents act like people. Pixels in, clicks out. If a machine can perceive the screen and click the buttons, that capability is the API. Batra told Chain of Thought that no one would pay developers to maintain custom scripts for 500,000 websites, while hundreds of thousands of browser agents can run in parallel.

2. Can a language model actually control a robot?

   Yes, and Google DeepMind's Paige Bailey describes it running on hobby hardware today: a Stanford-designed, 3D-printable robot dog taking spoken instructions through the Gemini APIs on a Raspberry Pi. What's new is the control layer, which became a general model you can talk to. Bailey told Chain of Thought the robot dog costs about $2,000 if you order the parts and 3D print it yourself, or about $2,500 preassembled.