Context engineering is deciding what an AI agent sees at each step: instructions, tool definitions, retrieved documents, earlier results and memory. Models reason less reliably when the window is crowded or stale, so load a small working set, fetch details on demand, summarise long histories and test the context like code.
Why the context window decides what an agent can do
A language model has no knowledge of your business beyond what it was trained on and what you put in front of it on each call. For an AI agent, that "in front of it" part is the context window: the system instructions, the descriptions of the tools it can call, the documents it has retrieved, the results of earlier steps and anything it has been told to remember.
Prompt engineering was about wording one instruction well. Context engineering is the wider job of assembling that whole working set, step after step, while the agent runs. Thoughtworks added it to its Technology Radar for this reason: the quality of what goes into the window now matters more than clever phrasing.
Larger windows have not made the problem go away. Practitioners describe "context rot": when raw data is dumped into a large window, the model's reasoning gets weaker, and details in the middle of a long context are more likely to be ignored. A bigger window lets you fit more in. It does not mean the model will use all of it well.
What competes for space in an agent's context
It helps to list everything that ends up in the window, because each part has a different owner and a different way of going wrong.
Instructions and policies
The system prompt sets the agent's role, its limits and the rules it must follow. These tend to grow as every incident adds another line. Long, overlapping instructions are harder for the model to apply consistently, so review them like code and remove rules that no longer earn their place.
Tool definitions
Every tool the agent can call carries a name, a description and a parameter schema. Connect a few integration servers and the tool list alone can crowd out the task. Clear, short descriptions help, as covered in how to design tools an AI agent can use well, but so does only loading the tools relevant to the current task.
Retrieved knowledge
Search results and documents pulled from your systems give the agent facts it cannot know otherwise. They are also where stale or conflicting information gets in. Keeping an AI agent's knowledge current is part of context engineering, because an out-of-date policy document in the window is worse than none.
Conversation and task history
Every tool call and its output stays in the window unless something removes it. On long tasks, this history is usually what fills the space, much of it no longer relevant to the next step.
Practical techniques that keep context useful
Load an index first, details on demand
Instead of loading every document and tool up front, give the agent a short index of what is available and let it fetch the specific item it needs. This approach, often called progressive disclosure, keeps the window small and makes it clearer to the model which information matters for the step in hand.
Trim and summarise tool output
A database query or API call can return far more than the agent needs. Filter tool responses to the fields that matter before they reach the model, and replace long earlier results with short summaries once they have been used. Keep the full output in your logs, not in the window.
Separate working memory from long-term memory
Notes the agent needs for the current task belong in the window. Facts it should keep across tasks belong in a store it can query. Mixing the two makes the window grow with every session. For work that runs over a long period, checkpoints and summaries let the agent resume without replaying everything, as described in running an AI agent on a task that takes days.
Give sub-agents clean, narrow contexts
When one agent hands a sub-task to another, pass only what that sub-task needs, not the full history. The sub-agent does better with a focused window, and the main agent receives a short result instead of a long transcript.
Keep restricted data out of the window
Anything in the context can end up in the agent's output or its logs. Decide which data an agent is allowed to see before you decide how to load it. Stopping agents reaching restricted data covers the access controls, and retrieved content should be treated as untrusted input, since it is a common path for injected instructions. See the AI agent attack surface.
Measure and test the context, not only the output
Context engineering is easy to get wrong silently. The agent still answers; it just answers worse. Make the context visible:
- Log what the model actually saw on each step, not only the final prompt template, so you can explain a bad decision afterwards.
- Track context size per step and per task. Growing context drives cost and delay, which links this work directly to controlling what an AI agent costs to run and making an AI agent respond faster.
- Add evaluation cases for context failures: a stale document, a conflicting instruction, a tool result with irrelevant noise. What AI agent evals catch explains how to keep such cases in a regression suite.
When an agent misbehaves, look at its context before blaming the model. Missing, stale, conflicting or irrelevant context is a frequent cause, and it is usually cheaper to fix than switching models.
If you are building agents on your own systems and want a second view on how their context is assembled, talk to CodeDTX.
Frequently asked questions
Is context engineering the same as prompt engineering?
No. Prompt engineering focuses on how one instruction is worded. Context engineering covers everything the model sees on each call while an agent runs: instructions, tool definitions, retrieved documents, earlier results and memory. It is closer to software design than copywriting, because the context is assembled by code at every step and needs the same review, logging and testing as any other part of the system.
Does a larger context window remove the need for context engineering?
No. A larger window lets more fit, but models still reason less reliably when the window is crowded with raw or irrelevant material, and details buried in the middle are more easily missed. Larger contexts also cost more and respond more slowly. A large window is useful headroom, but the agent still does better with a focused working set chosen for each step.
How do we know if poor context is causing an agent's mistakes?
Log the exact context the model received on the failing step and read it. Look for missing facts, out-of-date documents, instructions that contradict each other or tool output full of noise. Then build an evaluation case from that example and test whether fixing the context fixes the behaviour. If it does, the problem was context, not the model, and the case stays in your regression tests.
Who should own context engineering in a team?
The engineers who build the agent should own how context is assembled, because it lives in code: retrieval, tool loading, summarisation and memory. Domain owners still matter. They decide which documents are authoritative and which instructions apply, and they should review changes to both. Treat the system prompt and context rules as versioned artefacts with named owners, not text anyone can edit.
