Ask an AI to build a feature and it gives you the most plausible version of that feature, not the correct one for your app. Those are not the same thing, and the distance between them is where your day goes.
You have seen the symptom. A rule in your code looks wrong from the outside but is right on purpose, some deliberate constraint that encodes a decision a real person made for a real reason. The model reads it, decides it looks like a mistake, and confidently rewrites it back to the textbook version. Nothing errors, nothing looks wrong in review, and the logic is now subtly broken. It did not misread your code. It read the code perfectly and had no way to know your intent, so it filled in the average intent instead.
The gap is not the model malfunctioning. It is the model answering a different question than the one you asked. You want to know what this codebase needs here. It computes what a typical codebase does here, and hands you either answer with the same confidence. Once you see that substitution, the rest is mechanical: why it happens, and how to give the model enough of your context that its answer lands on your code instead of the average.
It predicts the next token, it does not retrieve the answer
A language model does one thing: given the text so far, it predicts the next token, then the next, then the next. Each step it produces a probability distribution over its whole vocabulary and draws one token from it, weighted by everything it saw in training. That is the entire engine. There is no table of facts it looks up, no internal model of your product it queries, no step where it checks its output against the truth. Even when a coding agent reads your files into the prompt, those files are not facts it now knows. They are just more tokens to predict from.
Here is the mechanism on a single token. Given a half-written function like def max_of(a, b): return, the model does not know how to finish it. It assigns a probability to every token that could come next:
The size of that wrong slice is not fixed. The model draws from this distribution, and only at temperature zero does it always take the single most probable token. Turn the temperature up and the unlikely tokens, the buggy one included, get picked more often. Coding tools tend to run at low temperature, which is part of why they feel fairly deterministic, but the thing underneath is still a bet, not a lookup.
So "the model is accurate" really means "the model's most probable guess is usually right." Accurate and correct feel like synonyms until the probable answer and the true answer diverge, and for the parts of your app that make it yours, they diverge constantly.
Its only memory is the context window and your code
The model has no memory of your product between sessions, and none that survives outside the current window. Every time it works on your code it starts from nothing and rebuilds what it needs from two sources: the context you hand it, and the code already in the repo. That is the whole world it reasons from.
Notice what is missing from that world. The reason a rule exists. The customer conversation that produced it. The bug six months ago that made you add the guard. The fact that a number is 7 and not 5 because of a contract, not a preference. None of that is in the code, and none of it is in the window, so the model cannot use it. It does not know your intent is absent. It fills the gap with the most likely intent and moves on with full confidence.
Anthropic says this about its own tool in plain words: every Claude Code session starts with a fresh context window. The persistence you think the model has is not in the model. It is text the tooling re-loads into the window each time, and you decide what that text says.
Plausible is the average app, not your app
Here is why your correct code gets flagged as a bug. The model's sense of what is normal comes from the distribution of everything it trained on, which is close to the average of all the public code ever written. When it evaluates your code, it is quietly comparing it against that average.
Your product is not the average. The whole point of building something is the specific, deliberate choices that make it different, and those choices are, by definition, rare patterns in the training data. Think of it less as a lookup and more as a strong prior: the model puts most of its probability on the patterns it saw most often. A rule that holds for your app and almost no other app sits far out on that prior, so the most probable continuation is the one that "corrects" it back toward the common case. It is not seeing an intentional constraint. It is seeing an outlier that looks like the normal pattern with a mistake in it.
The model is not wrong about the statistics. It is wrong about you, and it cannot be right about you from the prior alone. But a prior is only a default, and that is the good news: enough of the right context overrides it and pulls the model off the average onto your case. Which raises the practical question: where do you put that context so the model actually reads it?
Write the intent down where the model can read it
If the model only knows what is in its window, the highest-leverage move you have is controlling what goes into that window. Not per prompt, every session, automatically. Every serious AI coding tool now reads a project instruction file for exactly this, and most developers barely touch it. That file is where your product's intent becomes something the model can actually use.
The files, by tool:
- Claude Code reads
CLAUDE.md. It is loaded in full at the start of every session, and it layers: a global file in~/.claudefor your personal defaults, a projectCLAUDE.mdcommitted for the whole team, and an optionalCLAUDE.local.mdfor personal overrides. It can pull in other files with an@pathimport, so a project keeps one shared source of truth. Anthropic suggests keeping it under about 200 lines, because it competes for the same context as your code. AGENTS.mdis the cross-tool version. An open, plain-Markdown convention (agents.md) read by Codex, Cursor, Gemini CLI, Aider, Windsurf, and many others. As of 2026 it is used in tens of thousands of repositories and stewarded under the Linux Foundation. Claude Code does not read it natively, but you can bridge the two by importing it fromCLAUDE.md, which is exactly what this site's repo does.- Cursor uses
.cursor/rules/*.mdc, one scoped rule per concern with glob targeting. The older single.cursorrulesfile still works but is deprecated. - GitHub Copilot reads
.github/copilot-instructions.mdand adds it to your Copilot requests automatically.
One thing to be precise about, because it is the most common myth: none of this depends on a magic filename. There is no universal context.md that tools auto-discover. A file only reaches the model if something the tool already loads imports it, with an @ reference or similar. So the filename does nothing by itself; the import is what puts the content in front of the model.
Which frees you to split the document by concern, and that split is worth making once a project has real domain logic. Keep the instruction file (CLAUDE.md, AGENTS.md) for how to work in the repo: the conventions, the commands, the patterns. Put what the project actually is, and why, in a separate doc, a CONTEXT.md or an ARCHITECTURE.md, and import it. That second file is where the business-logic decisions live: why the limit is 7 and not 5, which rule looks wrong but encodes a real constraint, what the product is trying to be. It is exactly the intent the code cannot carry, and it is the highest-value thing you can hand the model, because it is the part the model can never infer. Every correction you find yourself repeating is a line that belongs in one of these files, so you never give it twice.
Package the repeatable parts as skills
A static instructions file covers what is always true about your project. A skill covers a repeatable workflow: a small Markdown file describing how to do one task, which the agent runs when you ask or reaches for when a task fits.
Matt Pocock keeps a public set at github.com/mattpocock/skills: TDD, bug diagnosis, turning a discussion into a PRD or issues, and more. Install them with npx skills@latest add mattpocock/skills.
One of them, handoff, maps directly onto this post. When a session's context window fills and quality drifts, it compacts the conversation into one document so a fresh agent can continue. It writes that document to your OS temp directory instead of the workspace, lists which skills the next agent should invoke, references existing artifacts by path instead of copying them, and redacts secrets.
That is the same move as the instructions file, one level up: when the model is about to forget, you decide what it keeps.
The job moved to owning the intent
Put it together and the shape of the work has changed. The model now does a large share of the typing, and the scarce thing is the part it structurally cannot do: knowing what the software is supposed to do, and why. That was always the hard part of engineering. It is now most of the part that is left.
In practice that is three habits, and this post has already covered the first two:
- Externalize the intent into the files and tests the tool reads, so it stops guessing at what only you know.
- Enforce what must hold with a test or type the build runs, since a written instruction cannot guarantee the model complies.
- Read every diff as a draft. A confident diff is a probability claim, not a correctness claim. The review that counts is not "is this good code," it almost always looks like good code. It is "is this what we actually intended," which only you can answer, because only you hold the input the model never got.
Getting the most from these tools is less about clever prompts than about holding a clear picture of what your software is for, and putting that picture where the model reads it. The model optimizes for plausible. Keeping it correct is your job.