Lesson 6.2 · 11 min read
Somewhere in your project there is a string. It's long, it's written in English, and it sits in the middle of a file, wrapped in quotes. It says something like: you are a helpful assistant, given the following support ticket, classify it into one of these categories, respond only with JSON.
Nobody thinks of that string as code. It looks like a note. It was probably written in five minutes, tweaked a few times until it worked, and forgotten.
Here's what it actually is. It is the logic of your feature. Change one sentence in it and the behavior of your product changes. Delete a phrase and the output format breaks. Reorder two instructions and accuracy shifts. That string determines what your system does, more than any function around it does.
You have logic in your codebase that nobody reviews, that nobody tests, that has no version history worth speaking of, that anyone can change without ceremony, and that is not obviously identifiable as the thing responsible when behavior changes. In any other part of your system this would be an emergency. Here it's Tuesday.
This post is about taking that seriously.
Before treating prompts as code, look at what's usually crammed into that one string, because it's rarely one thing.
There's the instruction: what you want done. Classify this. Summarize that. Extract these fields.
There's the role and constraints: who the model should be, what it must never do, what tone to take, what it should say when it doesn't know.
There's the output format: return JSON with these exact keys, use only these category names, respond in one sentence.
There are the examples: a few demonstrations of correct input and output, which are often the single most effective way to get reliable behavior, and which can be most of the length.
And there's the data: the actual support ticket, the user's message, the retrieved documents.
Those five things have completely different lifecycles, and mashing them into one string is the original sin. The data changes on every call. The examples change when you discover a failure mode. The format changes when the downstream code changes. The instruction changes when the product changes.
The first discipline, then, is separation. The data is not part of the prompt; it's an argument to the prompt. What you actually have is a template, a fixed structure with holes, and values that fill the holes at call time. That distinction sounds pedantic and it is the foundation of everything else in this post, because a template is a thing that can be stored, versioned, reviewed, and reused, and a string built by gluing text together at runtime is not.
It also closes a security hole, and you have seen this exact shape before. In the SQL post, you learned that gluing user input directly into a query lets a user change the query's meaning, and that the fix is passing input as a parameter rather than as instructions. The same attack exists here, and it's called prompt injection: a user whose text is pasted into your prompt can write text that reads like an instruction, and the model, which cannot reliably distinguish your instructions from content that resembles instructions, may follow it. The problem is harder than the SQL version and does not have a clean, complete solution. But keeping a firm separation between "this is the instruction I wrote" and "this is untrusted content from a user," and telling the model as much, is where every defense starts.
Take the ordinary situation. The prompt lives in a source file. What actually goes wrong?
You cannot change it without a deploy. Prompts change constantly, especially early. Someone notices the summaries are too long, or the classifier keeps choosing the wrong category for a certain kind of ticket. That's a one-line fix to English text, and it requires a code change, a review, a build, and a deployment. That friction is not neutral. It means fewer improvements get made, and it means the people best positioned to improve the prompt, the ones who understand the product and read the outputs, cannot do it without an engineer.
Nobody can see what changed. In principle the diff exists in Git. In practice, a change to a two-thousand-character string shows up as one enormous modified line, and the reviewer's eyes slide off it. The most consequential change in the pull request is the one nobody read.
You cannot answer the most important question in an incident. Something started behaving badly on Tuesday. Which version of the prompt produced that output? If prompts live in code, the answer requires correlating deploy times with logs, and if the prompt is assembled at runtime from several pieces, the answer may not exist at all. You cannot debug a system when you cannot reconstruct its inputs.
You cannot compare two versions honestly. You changed the prompt and it seems better. Better than what, on what inputs, by what measure? Without the old version running alongside on the same inputs, "seems better" is a feeling, and feelings about model output are notoriously unreliable, because you are looking at three examples you happened to check.
Each of these problems has the same root. The prompt is behaving like configuration and being treated like a code comment.
So what does it look like to do this properly? Four properties, and they're recognizable, because they're the properties you already demand of everything else in your system.
It is versioned, and the version is recorded with the output. Every prompt has an identifier and a version number. Every time the model is called, you log which prompt version was used, along with the model, the input, and the output. Now when something goes wrong on Tuesday, you can answer exactly what produced it. This single practice is the difference between a system you can operate and one you can only apologize for.
It is editable without a deploy. The prompt is stored somewhere your application reads it from at runtime, rather than compiled into the application. A change to the wording is a change to data, not to code. This is what makes fast iteration possible, and it puts the prompt in reach of the people who should be shaping it.
It is reviewed. Not in a diff of one giant line. As a document, read as a document, by someone asking whether these instructions are clear, whether the examples are right, whether the constraints still make sense. Prompt review is a genuinely different activity from code review, and it's closer to editing than to engineering, which is worth noticing given who is reading this.
It is evaluated. Before a new version replaces an old one, it is run against a set of saved inputs, and its outputs are compared. This is the thing that converts "seems better" into "is better on forty-three of fifty cases, and worse on two, and here is what those two look like." We'll come back to evaluation properly in the observability post, but understand that it depends entirely on versioning, because you cannot compare two things you cannot name.
Notice what these four are. Version control, separation of configuration from code, code review, and testing. They are the practices from Module 4, applied to a thing that happens to be written in English. Nothing here is exotic. What's exotic is that an entire industry skipped them because the artifact didn't look like code.
There is a category of tools for this, and they do roughly what you'd expect: store prompts, version them, let you edit them outside the codebase, run a new version against saved test cases, track which version produced which output, and let non-engineers participate.
I'm deliberately not going to build this post around any particular one. This part of the ecosystem is young and moving fast, and a post organized around a specific product would be worse than useless in eighteen months. More importantly, choosing a tool is not the point and never was. Teams manage prompts perfectly well in a database table with a version column. Teams manage them badly with expensive platforms.
What matters is that you can answer these questions about whatever you're doing:
Can I see every version this prompt has ever had, and who changed it, and why? For a given bad output in production, can I say exactly which prompt and which model produced it? Can I change the wording without a deploy, and roll it back if it's worse? Can I run a proposed change against real past inputs before it reaches a user?
If you can answer all five, your tooling is fine, whatever it is. If you can't answer them, no tool will save you, because the problem is that nobody decided these questions mattered.
Here's the part that should interest you most, and that most engineering-oriented writing on this topic misses entirely.
The prompt is where your product's voice is defined. It is where you decide what the system says when it doesn't know something, and whether it hedges or states plainly, and how it addresses the user, and what it refuses, and how it refuses. Every one of those is a design decision, made in prose, encoded in a string that a developer typed while thinking about output formats.
Consider the difference between a model instructed to always provide an answer and one instructed to say clearly when the available information is insufficient. That is not a technical parameter. It is a decision about the relationship between a product and the person using it, and it will do more to determine whether users trust your product than any interface you draw around it.
Or consider a summarizer told to produce three bullet points versus one told to write a short paragraph. Someone chose. Probably not deliberately, and probably not the person who thinks hardest about how the output will be read.
This is precisely the territory where a design engineer should be operating, and where the separation of prompt from code becomes not just an engineering hygiene issue but an organizational one. When prompts are buried in source files, only engineers touch them, and the product's voice is set by whoever was implementing the feature. When prompts are artifacts that can be read, discussed, and edited by the people responsible for the experience, the voice becomes something designed.
You have spent your career caring about the exact words on a button. The words in the prompt are more consequential than the words on any button in your product, and they were almost certainly written faster and reviewed less.
One last thing, and it's the one that makes people take this seriously.
Prompts are brittle in a way that code is not. A function refactored to be cleaner behaves identically. A prompt reworded to be clearer may behave differently, in ways nobody predicted, on inputs nobody tested. Adding a helpful clarification can degrade accuracy. Removing an apparently redundant sentence can break the output format on a rare input.
This means the ordinary intuition that improvements are safe does not hold. Every prompt change is a behavior change, and it must be treated as one: measured against real inputs, rolled out carefully, and reversible.
And the same brittleness applies from the other direction, which is the part people forget. Changing the model changes the behavior of every prompt. The provider releases a new version, you upgrade because it's faster and cheaper, and prompts tuned against the old one now behave subtly differently. Nothing in your code changed. Your product did.
This is why the versioning discipline is not bureaucratic. It's the only thing that lets you say "outputs got worse on the fourteenth, and on the fourteenth we changed from prompt version 7 to version 8 and stayed on the same model, so it was the prompt." Without that, you have a system whose behavior can change for reasons you cannot reconstruct, which is not an engineering artifact at all. It's a haunting.
Do this before the next post
Go find the prompts in something you've built, or in an open source project that uses a model, and read them as the artifacts they are.
First, locate them. If they're in the middle of a source file, notice how they look: a wall of English inside quotes, indistinguishable from a comment, unreviewable in a diff. Notice, too, whether you can tell how they've changed over time. Try to find out what that prompt looked like a month ago. Observe how hard that is.
Then take one prompt and separate it, on paper, into the five parts from the start of this post. Where is the instruction, the role and constraints, the output format, the examples, the data? Now ask which of those five change on every call, which change weekly, and which change once a year. Anything you find in the same string with wildly different lifecycles is telling you where the seams should be.
Finally, and this is the one worth actually doing: read the prompt as a piece of writing, out loud. Is it clear? Is it contradictory? Does it tell the model what to do when it doesn't know, or does it leave that unspecified so that the model will invent something? Does the voice it describes match the voice of the product?
You are extremely likely to find that the most important text in your application was written carelessly by someone in a hurry, and that you are better qualified to improve it than they were. That realization is the beginning of owning this layer, and the layer is more important than almost anyone building on it currently believes.