When you chat with an AI product, your message usually isn’t the first thing the model sees. Before it comes a system prompt written by whoever built the product: who the assistant is, what it should and shouldn’t do, which tools it can use, and how to format answers. Custom assistants and agents are largely defined by their system prompts.
A good system prompt reads like a brief for a capable new colleague: the goal, the audience, the constraints and what a good result looks like. It counts toward the context window and is billed as input on every call, so long ones add cost; that is one reason providers offer prompt caching. Prompt vs. context vs. memory engineering shows where the system prompt sits among everything else a model sees.
Hear it from the guest
“When we think about guardrails, it's typically like these prompts or these instructions that we can set in the system prompt for the model, but the model may choose to disregard those guardrails at some point”
Quotes lightly edited to remove filler words.
Go deeper
From the conversation
-
AI in 2025: Agents & The Rise of Evaluation-Driven Development | Galileo's Yash Sheth & Atindriyo Sanyal -
Slack Wants to Be the Context Harness for Code | CPO Jaime DeLanghe -
Thomson 1: The New $40M Legal AI Model | Thomson Reuters Joel Hron -
How Superhuman Built AI Into a 100ms Product | Loïc Houssier - I Built an AI Coworker That Runs 90% of My Day | Sterling Chin, Postman
- Architecting AI Agents: The Shift from Models to Systems | Aishwarya Srinivasan