Context engineering: why your agent gets dumber as its context grows
For multi-step agents the job shifted from writing the perfect prompt to curating the smallest high-signal set of tokens at every step. Because of context rot, more context literally makes agents worse.
There’s a moment every agent builder hits: the demo works for ten steps, then the agent starts repeating itself, forgetting what it decided, and burning tokens on a context that’s become a junk drawer. The instinct is to tweak the prompt. The real fix is upstream — it’s not what you say to the model, it’s what you let into its context window at all. That discipline is context engineering, and for agents it has largely replaced prompt engineering as the skill that matters.
More context can make the model worse
Anthropic defines context engineering as curating and maintaining the optimal set of tokens during inference — a deliberate step beyond prompt engineering’s focus on wording. The reason it matters is a phenomenon they call context rot: as the number of tokens in the window grows, the model’s ability to accurately recall information from it decreases, a consequence of the transformer’s quadratic pairwise attention “budget.” So a bigger context isn’t free headroom — past a point it actively dilutes the signal. The guiding principle becomes finding the smallest possible set of high-signal tokens that maximize the likelihood of the desired outcome. The full picture, with an interactive demo of a window filling and compacting, is in the context engineering lesson.
Three moves that keep the window lean
The toolkit is small and powerful. Compaction summarizes old turns and drops the raw history; in a long-horizon evaluation, context editing let agents complete 100-turn workflows that otherwise failed from context exhaustion, while cutting token use by about 84%. Isolation hands a sub-task to a specialized sub-agent working in a clean context that returns only a distilled result — often just a 1,000–2,000-token summary — so the main agent never sees the noise. And just-in-time retrieval keeps lightweight identifiers (file paths, queries, links) in context and loads the actual data at runtime via tools instead of pre-loading everything.
The extreme version: do the work outside the model
The most aggressive form of context engineering is to stop routing data through the model entirely. Representing MCP servers as code-level APIs the agent programs against — running tool calls and data processing in a sandbox — keeps the tool catalog and intermediate results out of the context window. The numbers are startling: one end-to-end workflow fell from roughly 150,000 tokens to about 2,000 — a 98.7% reduction — when converted to filesystem-based APIs plus a code-execution loop. Same task, a fraction of the tokens, because the bulk of the work happened where the model couldn’t see it. That technique gets its own code-execution-with-MCP lesson.
Prompt engineering isn’t dead, but for anything that runs more than a couple of steps it’s the wrong altitude. The agents that stay sharp over long horizons are the ones whose builders obsess over what’s in the window — compacting, isolating, and retrieving just in time. Start with the context engineering lesson and you’ll never look at a growing transcript the same way again.