Prompt Management

A product manager edits the onboarding assistant’s prompt to sound friendlier. If the prompt is a string in the code, that needs an engineer and a deploy, and nobody can easily tell later which wording was live when a complaint came in. With prompt management, the prompt lives in a registry as version 14. The manager saves version 15, it runs against the regression suite, gets labeled “staging”, and is promoted to “production” only if it passes. If users react badly, production points back at version 14 in seconds.

A prompt registry usually records the template, its variables, the model and settings it was tested with, who changed it and why. Versions are immutable, and labels or aliases decide which version each environment uses. Logged requests and LLM traces store the version they ran with, so quality changes can be tied to a specific edit, and A/B tests can compare two versions on live traffic.

Pulling prompts out of the code is a trade-off. It lets non-engineers iterate, but it also creates a path for changing production behavior without a code review. Treat prompt promotion as a release, with tests and approval, and restrict who can promote. A prompt also depends on its model, so re-test when either one changes.

In episode 59 at 20:02, Superhuman CTO Loïc Houssier describes a platform team that gives feature teams prompt management, versioning and evals as shared infrastructure, alongside an internal model router, so each product team can move independently. Episode 59 was sponsored by Galileo; the read is at 1:22.

See prompt engineering for writing prompts and LLMOps for operating the whole application.