Skip to content

AI Glossary · Concepts

What is System Prompt?

System Prompt

A system prompt is the standing set of instructions given to a language model before the user's message, setting its role, rules, tone and output format for the whole conversation.

Also known as: system prompts, system message

· Updated · Chain of Thought

Context Management

  • 2 explainers
  • 6 episodes
  • 1 guest clip
  • 2 sources

When you chat with an AI product, your message usually isn’t the first thing the model sees. Before it comes a system prompt written by whoever built the product: who the assistant is, what it should and shouldn’t do, which tools it can use, and how to format answers. Custom assistants and agents are largely defined by their system prompts.

A good system prompt reads like a brief for a capable new colleague: the goal, the audience, the constraints and what a good result looks like. It counts toward the context window and is billed as input on every call, so long ones add cost; that is one reason providers offer prompt caching. Keep credentials out of it: system prompts can often be extracted, which is one route to token leakage. Prompt vs. context vs. memory engineering shows where the system prompt sits among everything else a model sees.

Price a long system prompt

Anthropic’s prompt caching documentation gives the multipliers for a worked example. Take a 5,000-token system prompt sent with 100 requests. Without caching, that is 100 × 5,000 = 500,000 input tokens at the base price.

With the default 5-minute cache, the first request writes the cache at 1.25 times the base price, the equivalent of 6,250 tokens. If each later request arrives while the cache is still alive, the other 99 read it at 0.1 times: 99 × 500 = 49,500. The total is 55,750 token-equivalents, about 11% of the uncached cost. If requests arrive more than 5 minutes apart, every one writes the cache again: 100 × 6,250 = 625,000, which is 25% more than not caching at all.

The arithmetic depends on the details: a few models read the cache at a lower multiplier, the 1-hour cache writes at twice the base price, and prompts below a model’s minimum length (512 to 4,096 tokens, depending on the model) aren’t cached. Check the current figures for your model. Caching changes what the instructions cost. It doesn’t change what the assistant is allowed to do.

Hear it from the guest

“When we think about guardrails, it's typically like these prompts or these instructions that we can set in the system prompt for the model, but the model may choose to disregard those guardrails at some point”

Quotes lightly edited to remove filler words.

Sources

  • OWASP: LLM07:2025 System Prompt Leakage — Says a system prompt should not be treated as a secret or used as a security control, so credentials and connection strings do not belong in it; checked October 25, 2026.
  • Anthropic: Prompt caching — Explains caching repeated prompt prefixes, such as long standing instructions, to cut cost and latency on later calls. Lists a 5-minute default cache lifetime refreshed on each use, cache writes at 1.25 times the base input price, cache reads at 0.1 times for most models, and per-model minimum cacheable lengths; checked November 7, 2026.

Go deeper

From the conversation