Prompts drift. Someone tweaks a line to fix one bad case, ships it inline, and three weeks later no one can say what the prompt used to be or why quality dropped. The fix is to treat a prompt like any other released artifact: versioned, reviewed, evaluated, and reversible.
Pin the whole unit
The output of an LLM feature is determined by the prompt and the model and the sampling parameters. Versioning the prompt text alone is not enough — a model upgrade or a temperature change alters behaviour just as much. Version them together as one config object so a "prompt version" fully describes the behaviour it produces.
PROMPTS = {
"classify@3": {
"model": "claude-sonnet",
"params": {"temperature": 0, "max_tokens": 256},
"system": "You are a support ticket classifier. ...",
},
}
def render(name):
spec = PROMPTS[name]
return spec # model + params + text travel togetherKeep prompts in version control, not a database row someone can edit unlogged. A pull request diff on a prompt is exactly the review you want: a teammate can see the change, and git blame tells you when and why each line arrived.
Evaluate before you promote
A new prompt version is a candidate, not a release. Run it against your eval dataset and compare the aggregate score to the current production version. Promote only if it wins — or at least does not regress your guardrail cases. This is the discipline that stops the "fixed one case, broke five" pattern that plagues hand-tuned prompts.
Changelog and attribution
Record what changed and why with each version bump, and stamp the prompt version onto every logged request. When quality shifts in production, the first question is always "what changed" — and the answer should be a version string you can look up, not an archaeology dig through chat history.
Put the prompt version in your cost logs, your eval results, and your request traces. That one identifier lets you say 'spend rose and quality fell after classify@4 shipped on Tuesday' — which turns a vague quality complaint into a specific, reversible change.
Once prompts are versioned artifacts, everything else becomes possible: A/B testing two versions, rolling back a bad one in seconds, and answering "what is running in production right now" with certainty instead of a guess.
Independent software engineer in Nairobi specialising in Acumatica customisations, Laravel backends, and tax fiscalisation integrations across East and Southern Africa.