Test retrieval on its own before you judge the answers of a RAG agent. Build a fixed set of real questions, each paired with the documents or passages that should come back, and score the search step against it. If the right passages never reach the model, no prompt change or model upgrade will fix the answer.
Why the search step hides most RAG failures
A retrieval-augmented agent does two jobs: it finds material, then it writes from that material. When a user reports a wrong answer, teams usually look at the second job first. They rewrite the prompt, add instructions or try a different model. Often the real problem sat one step earlier: the passage holding the answer was never retrieved, or it was ranked below a dozen near misses and cut off.
Research supports looking there first. The study Understanding the Fundamental Design Decisions of Retrieval-Augmented Generation Systems found that retrieval recall is a dominant factor in how well a RAG system answers. A model can only use what it receives.
In an agent the cost is higher than in a chat assistant. An agent that retrieves the wrong policy and then acts on it carries the mistake into every later tool call. That is why retrieval deserves its own test, separate from the evals that check what an agent does.
Build a retrieval test set from real questions
The test set matters more than the choice of metric, and it is the part teams most often skip.
Start with questions people actually ask
Pull questions from support tickets, search logs, pilot transcripts and the people who will use the agent. Synthetic questions generated by a model can widen coverage, but they should add to a core of real ones, not replace it. Have a person read every question before it goes in.
Label the passages that should come back
For each question, record the documents or chunks that contain the answer. This is slow work and needs someone who knows the subject. Where more than one passage is acceptable, list them all, so the test does not punish a correct alternative.
Version the set like code
Save the set as a named, versioned file. When documents change or new questions arrive, create a new version rather than editing in place. Otherwise last month's scores and this month's scores measure different things.
What to measure
Keep the measures few and easy to explain.
Recall: did the right passage come back at all
Recall asks whether the labelled passages appear anywhere in the results the agent passes to the model. For RAG it is usually the measure that matters most, because a missing passage cannot be recovered later.
Ranking: did it come back near the top
Ranking measures such as mean reciprocal rank show whether the right passage sits near the top of the list or near the cut-off. Poor ranking wastes context space and makes the model more likely to quote a near miss instead.
Precision: how much noise came with it
Precision shows how much of what was retrieved is relevant. Low precision fills the context window with distracting material, which ties into how you decide what goes in the window.
Run the tests whenever the search changes
Retrieval quality changes for reasons that have nothing to do with the model:
- New documents added, old ones removed or rewritten
- A different chunk size or splitting rule
- A new embedding model or a re-built index
- Changes to filters, permissions or metadata
- A new re-ranking step
Run the retrieval tests on every one of these, the same way unit tests run on a code change. Keeping an agent's knowledge current is only safe if you can see when an update made search worse.
Offline tests miss some things. Sample real production queries, check what was retrieved, and add the failures to the next version of the test set. Pair this with monitoring the agent in production so a drop in retrieval shows up before users notice it.
When to fix retrieval and when to fix the model
Once retrieval has its own score, a wrong answer becomes easier to place. If the right passage was not retrieved, work on search: chunking, embeddings, filters, query rewriting or hybrid keyword and vector search. If it was retrieved and ranked well but the answer is still wrong, the problem sits with the prompt, the model or the instructions on how to use sources.
This split saves time and money. Teams stop swapping models to fix a search problem, and stop tuning search to fix a reasoning problem. CodeDTX builds this kind of measurement into the AI integration work we do, so a client can see where an answer went wrong. If your RAG agent gives answers you cannot explain, talk to us.
Frequently asked questions
How many questions does a retrieval test set need?
Enough to cover the main kinds of question your users ask and the main document types the agent searches. Start small with questions a person has read and labelled, then grow the set from real production failures. A smaller set of carefully labelled questions tells you more than a large set of unchecked synthetic ones, because a wrong label makes the score itself wrong.
Can a model judge retrieval quality instead of a person?
A model can help, for example by checking whether retrieved passages contain the facts a reference answer needs. It is useful for scale and for questions with no single correct passage. It should not replace human labels entirely. Check the judge against a sample of human-labelled questions first, and recheck it whenever you change the judging model or its prompt.
Should retrieval tests block a release?
For an agent that acts on what it retrieves, yes. Agree a recall threshold for the questions that matter most, such as policy, pricing or safety questions, and fail the release if a change drops below it. For lower-risk questions, a drop can raise a warning for review instead. The aim is that nobody ships a worse search step without knowing it.
Does better retrieval remove the need for answer evals?
No. Retrieval tests show whether the agent received the right material. Answer evals show whether it used that material correctly, cited it, and took the right next step. A RAG agent needs both. Measuring them separately is what lets you tell a search fault from a reasoning fault, instead of seeing only that the final answer was wrong.
