What is the difference between retrieval and reranking in a RAG pipeline?

How to think about it

Retrieval is the cheap, broad first search over the corpus. Reranking is the more expensive second pass over the retrieved candidates, where a model looks closely at each query-document pair and improves the ordering. The crucial limitation is simple: a reranker cannot rescue a relevant document that the first stage never returned.

Why the two stages exist

A retrieval-augmented generation, or RAG, system searches external information and gives selected passages to a language model before it writes an answer. The system might have 10 million document chunks, but the generator cannot sensibly read all of them for every question. It needs a small, relevant context.

The pipeline therefore has a funnel:

Stage	Sees	Produces	Main objective
Retrieval	The full corpus	A candidate set, often 50 to 500 chunks	High recall
Reranking	Only those candidates	The best-ordered shortlist	High precision at the top
Generation	The shortlist	An answer, usually with citations	Grounded response

Recall means avoiding misses. If the correct chunk is anywhere in the candidate set, retrieval has succeeded for that query. Precision means putting useful results near the top; if the final eight chunks contain six relevant ones, precision at eight is strong.

Retrieval is designed for scale. A BM25 retriever uses an inverted index to find documents containing useful terms. A dense retriever converts the query and documents into vectors, or numerical representations, and searches for nearby vectors. Because document vectors can be computed before the user asks a question, a vector index can search millions of chunks without running a large neural model against every query-document pair.

Hybrid retrieval combines lexical and dense methods. That matters because they fail differently. BM25 is good at exact identifiers such as ERR_CONNECTION_RESET, product codes, and legal phrases. Dense retrieval is better when the user says “I forgot my password” and the document says “credential recovery.” Combining them often gives the candidate stage a wider safety net.

Reranking optimizes the smaller set. A common reranker is a cross-encoder, a model that reads the query and one candidate document together rather than encoding them independently. Joint attention lets it notice details such as:

• The query asks about cancelling a subscription, while the document only explains pausing one.
• The query says “after renewal,” while the document applies only before renewal.
• The document contains every keyword but answers a different question.

That extra inspection is valuable, but expensive. A cross-encoder must process each candidate pair at query time. Scoring 200 candidates is practical. Scoring 10 million is usually not.

A concrete example

Suppose a company has 2.4 million chunks from product manuals, support tickets, and policy pages. A customer asks:

Can I carry over unused parental leave after moving from contractor to employee?

The dense retriever finds the relevant policy chunk at rank 42. BM25 misses it because the policy uses “employment conversion” rather than “moving from contractor to employee.” A hybrid candidate set contains 160 unique chunks after combining both searches.

An illustrative latency budget might look like this:

Operation	Work	Example budget
BM25 search	Search the inverted index	20 milliseconds
Dense search	Search the vector index	35 milliseconds
Candidate merge	Remove duplicates and apply filters	5 milliseconds
Reranking	Score 160 query-document pairs	100 milliseconds
Context selection	Keep the best eight chunks	1 millisecond

The numbers are an engineering example, not a universal benchmark. Hardware, model size, document length, batching, and network calls can change them substantially.

Before reranking, the relevant chunk is merely one item among 160. The reranker reads the question and each candidate, notices that the policy discusses employment conversion and leave eligibility, and moves the chunk to rank 3. The generator receives the top eight rather than all 160.

Now change the example slightly. The relevant policy is not in the hybrid set at all because it was never indexed, was excluded by a faulty tenant filter, or was split into chunks with no useful surrounding context. The reranker still scores 160 documents. It may produce an impressively ordered list, but the answer remains unavailable. Better sorting is not broader search.

Common misconception: A reranker can improve the relevant document’s position, but it cannot increase the set of documents available to the pipeline. If the document is absent at retrieval, the problem is retrieval.

How I would evaluate the stages

Do not evaluate only the final generated answer. That hides where the system failed.

For first-stage retrieval, use recall@100: the percentage of evaluation questions for which at least one gold chunk, meaning a chunk judged to contain the needed information, appears in the first 100 retrieved candidates.

Imagine 1,000 labeled questions:

• 920 questions have a gold chunk somewhere in the first 100 candidates.
• Retrieval recall at 100 is therefore 92 percent.
• Of those 920 questions, the reranker puts a gold chunk in the final eight for 805 questions.
• Conditional reranking success is about 87.5 percent.
• Overall, the final context contains a gold chunk for 805 of 1,000 questions, or 80.5 percent.

Those numbers tell different stories. The 80 missing cases are retrieval failures. The 115 cases where retrieval found the answer but reranking failed are ranking or selection failures. A metric such as mean reciprocal rank, or MRR, can also measure how early the first relevant result appears. For a RAG system, the most useful cutoff is often the number of chunks actually passed to the generator, not an arbitrary search result page.

I would log document IDs and ranks after every stage:

1. Which documents did lexical retrieval return?
2. Which did dense retrieval return?
3. What survived filtering and deduplication?
4. What ranks did the reranker assign?
5. Which chunks actually entered the prompt?

That makes an end-to-end quality drop diagnosable rather than mysterious.

The nuance that earns the senior signal

“Retrieval” is sometimes used as an umbrella term for the whole search subsystem, including reranking. In an interview, I would state the boundary explicitly: I mean initial candidate generation when I say retrieval, and query-dependent ordering when I say reranking.

The candidate count is a trade-off. Returning only 20 chunks may be fast but miss relevant evidence. Returning 1,000 improves the chance of inclusion but increases reranker cost, memory use, and duplicate or distracting content. A practical approach is to plot retrieval recall as the candidate count rises from 50 to 100, 200, and 500, then choose the point where extra recall is no longer worth the latency.

A reranker is not automatically an improvement. It can hurt when it was trained on a different domain, when documents are unusually long and get truncated, or when the model prefers fluent explanatory text over the authoritative policy. Freshness, permissions, language, product, and tenant boundaries should be handled with reliable filters and metadata. Do not expect a relevance model to learn access control.

For a small corpus, very low-latency application, or exact lookup problem, a separate reranking stage may not be worth its cost. If the corpus has 300 documents and a database filter already identifies the right product and version, a carefully designed search may be enough. Conversely, a large support corpus with ambiguous questions is a strong case for retrieval followed by reranking.

A failure mode I would debug first

The first visible symptom is often a confident answer citing an old policy while a newer policy exists.

I would check whether the newer chunk was in the initial candidate set. If it was missing from recall at 100, I would inspect indexing, chunk boundaries, metadata filters, query rewriting, and the lexical or embedding retriever. If it was present at rank 73 but absent from the final eight, I would inspect the reranker, document truncation, score thresholds, and freshness handling. If it reached the final context and the answer was still wrong, retrieval and reranking are no longer the primary suspects; context assembly, instructions, or generation needs attention.

What they’ll ask next

Can a reranker increase recall?

It can improve recall at the final cutoff by moving an already-retrieved relevant chunk into the top eight. It cannot improve corpus-level recall, because it never sees documents outside the candidate set.

Why not rerank the whole corpus with a cross-encoder?

The cross-encoder must jointly process the query with every document. That means millions of query-document evaluations per question, instead of one vector search followed by a few hundred detailed evaluations. The two-stage design spends expensive computation only where it has a chance to affect the answer.

How do you choose the candidate count?

Measure recall at several candidate sizes and measure p95 latency, meaning the response time below which 95 percent of requests finish. Choose a size that leaves a safety margin for recall while meeting the product’s latency budget. Then test separately for short questions, long questions, rare identifiers, and filtered tenants; one average number can conceal a nasty edge case.

Say this in the interview

“Retrieval searches the whole corpus cheaply for high recall; reranking spends more compute ordering that candidate set for top-k precision, and it cannot recover a relevant document that retrieval missed.”