Ask a travel assistant for flights to Lisbon, then say “make it Porto instead,” then “and add a hotel.” Each message only makes sense with the earlier ones. The model has no memory between calls, though. Anthropic’s API documentation says it plainly: the API is stateless, so the application sends the whole conversation history with every request (Anthropic, Using the Messages API). What feels like memory in a chatbot is the app replaying the transcript.
That design has costs. Every turn resends everything before it, so long conversations use more tokens, take longer and eventually hit the context window. Apps manage this by trimming old turns, summarizing them with context compaction, or storing facts in agent memory. In episode 38 at 25:18, Aurimas Griciūnas argues that for agent builders this makes context engineering unavoidable, because “the context window is just exploding if you are building systems of multi turn conversations.” Conor plugs Galileo, then the show’s sponsor, at 28:13.
Quality changes too. In a large simulation study, leading models did markedly worse when a task was revealed over several turns than when it was stated all at once, with an average 39% drop across six tasks; models often guessed early and then clung to that guess (Laban et al.). That is why single-question tests can overstate how well a conversational product works. In episode 57 at 15:01, Snorkel AI’s Alex Ratner describes multi-turn evaluation as tasks where you “have multiple turns with the user” whose goals shift along the way. Galileo was the presenting sponsor of episode 57 (opening read).
Practical advice: test whole conversations, not just first messages, and when a chat has gone off track, a fresh conversation with a clear restated request often works better than more corrections.
Hear it from the guest
“The context window is just exploding if you are building systems of multi turn conversations”
“Have multiple turns with the user”
Quotes lightly edited to remove filler words.
Sources
- Anthropic: Using the Messages API — States that the Messages API is stateless, so you always send the full conversational history, and that earlier turns need not come from the model itself; checked October 24, 2026.
- Laban et al.: LLMs Get Lost In Multi-Turn Conversation — Simulation study finding top LLMs performed significantly worse in multi-turn than single-turn settings, an average drop of 39% across six generation tasks; checked October 24, 2026.
- Chain of Thought episode 38 at 25:18 — Aurimas Griciūnas says context engineering is unavoidable for agents because multi-turn conversations make the context window explode.
- Chain of Thought episode 57 at 15:01 — Snorkel AI's Alex Ratner describes multi-turn evaluation, where tasks require several turns with a user whose objectives shift.
