When you chat with an AI product, your message usually isn’t the first thing the model sees. Before it comes a system prompt written by whoever built the product: who the assistant is, what it should and shouldn’t do, which tools it can use, and how to format answers. Custom assistants and agents are largely defined by their system prompts.
A good system prompt reads like a brief for a capable new colleague: the goal, the audience, the constraints and what a good result looks like. It counts toward the context window and is billed as input on every call, so long ones add cost; that is one reason providers offer prompt caching. Keep credentials out of it: system prompts can often be extracted, which is one route to token leakage. Prompt vs. context vs. memory engineering shows where the system prompt sits among everything else a model sees.
Price a long system prompt
Anthropic’s prompt caching documentation gives the multipliers for a worked example. Take a 5,000-token system prompt sent with 100 requests. Without caching, that is 100 × 5,000 = 500,000 input tokens at the base price.
With the default 5-minute cache, the first request writes the cache at 1.25 times the base price, the equivalent of 6,250 tokens. If each later request arrives while the cache is still alive, the other 99 read it at 0.1 times: 99 × 500 = 49,500. The total is 55,750 token-equivalents, about 11% of the uncached cost. If requests arrive more than 5 minutes apart, every one writes the cache again: 100 × 6,250 = 625,000, which is 25% more than not caching at all.
The arithmetic depends on the details: a few models read the cache at a lower multiplier, the 1-hour cache writes at twice the base price, and prompts below a model’s minimum length (512 to 4,096 tokens, depending on the model) aren’t cached. Check the current figures for your model. Caching changes what the instructions cost. It doesn’t change what the assistant is allowed to do.
Hear it from the guest
“When we think about guardrails, it's typically like these prompts or these instructions that we can set in the system prompt for the model, but the model may choose to disregard those guardrails at some point”
Quotes lightly edited to remove filler words.
Sources
- OWASP: LLM07:2025 System Prompt Leakage — Says a system prompt should not be treated as a secret or used as a security control, so credentials and connection strings do not belong in it; checked October 25, 2026.
- Anthropic: Prompt caching — Explains caching repeated prompt prefixes, such as long standing instructions, to cut cost and latency on later calls. Lists a 5-minute default cache lifetime refreshed on each use, cache writes at 1.25 times the base input price, cache reads at 0.1 times for most models, and per-model minimum cacheable lengths; checked November 7, 2026.
Go deeper
From the conversation
-
AI in 2025: Agents & The Rise of Evaluation-Driven Development | Galileo's Yash Sheth & Atindriyo Sanyal -
Slack Wants to Be the Context Harness for Code | CPO Jaime DeLanghe -
Thomson 1: The New $40M Legal AI Model | Thomson Reuters Joel Hron -
How Superhuman Built AI Into a 100ms Product | Loïc Houssier -
I Built an AI Coworker That Runs 90% of My Day | Sterling Chin, Postman -
Architecting AI Agents: The Shift from Models to Systems | Aishwarya Srinivasan