What is the difference between retrieval and reranking in a RAG pipeline?
The short answer
Retrieval cheaply searches a large corpus and returns a candidate set, prioritizing recall. Reranking applies a more expensive query-document model to that small set and improves precision and ordering at the top. A reranker cannot recover relevant documents absent from the retrieved candidates, so evaluate first-stage recall separately.
How to think about it
Retrieval reduces millions of documents to perhaps 50–200 candidates using BM25, embeddings or hybrid search. Its job is high recall at manageable cost.
Reranking spends more compute on those candidates. Cross-encoders jointly inspect the query and document, which often improves top-k precision but does not scale to the full corpus.