Context Engineering for AI Agents: Practical Patterns

Context engineering is the practice of curating everything that goes into a model's context window (the fixed token budget a model can read at once, where a token is a chunk of text very roughly three-quarters of a word) on each turn of an agent's loop. That includes the system instructions, the tool definitions, any worked examples, the running conversation, retrieved documents and the agent's own memory.

It is a broader idea than prompt engineering, which is about wording a single request well. Prompt engineering asks "how do I phrase this question?" Context engineering asks "what should the model be able to see when it answers, and what should it not?" For a one-shot chatbot reply the two overlap. For an agent that runs dozens of steps, calls tools and accumulates history, context engineering is the larger lever - and it is why a well-fed average model routinely outperforms a frontier model working from a cluttered prompt. I have made the strategic case for that separately in why context engineering beats picking a smarter model; this piece is the hands-on companion.

Every step an agent takes adds tokens. A single tool call appends the request, the raw result (often verbose JSON), and the model's reasoning about it. Run an agent for twenty steps and you can burn six figures of tokens on plumbing before the model has done anything useful.

Two problems follow. The first is cost and latency: you pay for every token read, on every step. The second is subtler and more dangerous - models attend less reliably to information buried in the middle of a long context, a failure mode documented in the research literature as the lost-in-the-middle effect and one I have written about as context rot. A bigger window does not make this go away. It just lets you make the mistake at greater scale.

It helps to break the context into the five components you control. Get each one right and most agent failures disappear.

Tool descriptions are the most under-rated surface in agent design. The model decides whether and how to call a tool almost entirely from its name and description, so write them for the model, not for a human skim-reader.

Three rules carry most of the weight. State when to use the tool, not just what it does - 'use this to look up a live order status by ID; do not use it for historical orders older than 90 days'. Spell out each parameter's format and constraints inline. And describe the shape of what comes back, so the model knows what it can chain next. Anthropic's tool-use documentation and OpenAI's function-calling guide both make the same point: the description is part of the prompt, and ambiguous descriptions are the single most common cause of wrong tool calls.

The instinct with a large context window is to paste everything in - the whole knowledge base, the full document, every prior message. Resist it. Retrieval almost always beats stuffing once your reference material is larger than a few thousand tokens, for three reasons: you keep the high-signal-to-noise ratio that protects attention, you cut cost on every step, and you can update the knowledge without re-engineering the prompt.

The decision rule I use: if the information is small, stable and needed on every step, put it in the system prompt. If it is large, changes often, or is only relevant to some queries, retrieve it on demand. I cover the mechanics in what is RAG, but the context-engineering point is simply this: retrieval is how you keep a context window focused even when the underlying knowledge is enormous.

This is the question that trips people up, because the right answer inverts as the budget grows. Today's frontier models from Anthropic, Google and OpenAI offer context windows up to a million tokens or more, and it is tempting to assume that makes context engineering obsolete. The opposite is true - it changes which problem you are solving.

At around 1k-8k tokens, the constraint is space. You hand-curate ruthlessly: tight instructions, one or two examples, only the most relevant retrieved snippet. Every token is contested, so you spend your effort on compression.

At around 100k tokens, space stops being the binding constraint and attention becomes one. You can fit a lot, but the lost-in-the-middle effect means the model will not weight all of it equally. Now you spend your effort on ordering and structure - putting the most important material at the start and end, using clear headings, and summarising stale history rather than carrying it verbatim.

At 1M tokens, you can fit an entire codebase or a book, but two new problems appear: cost and latency scale with what you load, and precision still matters because more irrelevant text means more chances to distract the model. The winning pattern at this scale is not 'load everything' - it is 'retrieve aggressively, then use the huge window as headroom for the genuinely relevant material plus room to reason'. The window grew; the discipline did not change.

Memory is where agents most visibly fall apart, because the naive approach - keep appending the full history - guarantees you hit the context limit and the lost-in-the-middle wall at the same time. The fix is to treat memory as a managed resource rather than an ever-growing log.

Three patterns do most of the work. Summarise older turns into a compact running state so the agent keeps the decisions without the transcript. Externalise durable facts - user preferences, task progress, learned constraints - into a store the agent reads from and writes to, rather than holding them in the conversation. And scope each step to only the memory it needs, rather than re-reading everything every time. I unpack the failure mode in why your AI keeps forgetting things; the engineering answer is that a good agent decides what to remember as deliberately as it decides what to do.

The same handful of errors come up again and again. Dumping raw tool output (especially full JSON) straight back into the context instead of extracting the relevant field. Writing tool descriptions for humans rather than the model. Carrying the entire conversation history verbatim when a summary would do. Pre-loading a knowledge base that should be retrieved on demand. And treating a bigger context window as permission to stop curating. Each one quietly degrades the agent while leaving the model itself untouched - which is exactly why blaming the model is so tempting and so wrong.