Vectorless retrieval (PageIndex)
Retrieval without embeddings — PageIndex turns a document into a tree the LLM reasons down. When it beats a vector database, and when it doesn't.
What you'll learn
- How vectorless (tree-navigation) retrieval works, step by step
- Why structure-aware retrieval can beat flat top-k similarity
- The real trade-off — LLM calls per hop vs millisecond ANN at scale
- When to choose PageIndex, a vector DB, or both together
Before you start
Standard RAG has one reflex: embed everything, embed the query, retrieve the nearest chunks. It’s fast and it scales — but it’s flat. It throws away the document’s structure and retrieves by surface similarity, which can miss the right passage while grabbing plausible-looking neighbours. PageIndex is a “vectorless” alternative that keeps the structure and lets the model reason.
The idea: a tree the LLM can navigate
PageIndex preprocesses a document into a hierarchical tree — exactly its table of contents. Each node carries a title, a short summary, and the page range where its content lives. No embeddings, no vector index.
At query time, the LLM is handed the tree and navigates it like a human flips to a chapter: at each level it reasons about which branch is most relevant, descends, and repeats until it reaches the right section — then reads just that section’s pages to answer. Toggle the two approaches below on the same query:
PageIndex: the LLM reads the table of contents and reasons a path down — root → Financials → Risk factors — then reads just that section. Structure-aware and explainable, but it spends an LLM call per hop, so it suits bounded, well-structured documents.
Not a winner-takes-all: vector search for scale and speed; vectorless tree-navigation for reasoning over structured docs. Many systems combine them.
The contrast is the whole lesson. Vector search finds the right section and a
couple of unrelated chunks, because cosine similarity is blind to where things
sit in the document. PageIndex walks root → Financials → Risk factors by
reasoning — and can explain why it went there.
The trade-off, honestly
Vectorless retrieval is not strictly better — it trades a different set of costs.
| Vector search | PageIndex (vectorless) | |
|---|---|---|
| Index | embeddings + ANN (HNSW) | a parsed document tree |
| Query cost | one ANN lookup, milliseconds | one LLM call per hop down the tree |
| Scale | millions of chunks, cheap | best for bounded sets of structured docs |
| Strength | speed, recall at scale | structure-awareness, reasoning, explainability |
| Weakness | flat; ignores document structure | slow + costly per query; needs clean structure |
Why this exists now
Two things made vectorless retrieval practical: models got good enough at multi-step reasoning to navigate a tree reliably, and long context windows mean a node’s summaries (and the chosen section) fit comfortably in the prompt. It’s part of a broader shift — using the model’s reasoning at retrieval time instead of treating retrieval as a pure similarity-lookup problem.
Check yourself
Quick check
What to remember
- PageIndex is vectorless retrieval: build a tree (titles, summaries, page ranges) and let the LLM reason a path down to the right section.
- It’s structure-aware and explainable, where flat vector top-k is fast but blind to document structure.
- The cost is an LLM call per hop — great for bounded, structured documents, not for million-chunk corpora.
- Vector search and PageIndex aren’t rivals: shortlist with vectors, then navigate within a document with PageIndex.
Questions about this lesson
What is vectorless retrieval?
Vectorless retrieval finds relevant content without embeddings or a vector index. PageIndex is the leading example: it turns a document into a hierarchical tree (titles, short summaries, and page ranges) and the LLM reasons a path down it — choosing the most relevant branch at each level, like flipping to the right chapter — then reads only that section to answer. Retrieval becomes a reasoning task over structure rather than a similarity lookup.
Is PageIndex better than a vector database?
Neither is universally better; they trade different costs. A vector database does one approximate-nearest-neighbour lookup in milliseconds and scales to millions of chunks, but it's flat and ignores document structure. PageIndex is structure-aware and explainable but spends an LLM call at each hop down the tree, so it's slower and pricier per query and best for a bounded set of well-structured documents. Many systems combine them: shortlist documents with vectors, then navigate within one using PageIndex.
When does vectorless retrieval beat flat vector RAG?
When the document has clear structure and the answer is found by reasoning about where it lives rather than by surface similarity. Flat top-k can return plausible-but-wrong neighbours because cosine similarity is blind to structure; navigating a tree to 'Financials → Risk factors' is more precise and can explain its choice. For huge unstructured corpora where speed and recall dominate, vector search still wins.
Practice this in an interview
All questionsVectorless retrieval skips embeddings entirely: it organizes a document into a hierarchical tree (titles, summaries, page ranges) and the LLM reasons a path down it — root to chapter to section — then reads only the chosen section to answer. It is structure-aware and explainable, but it spends an LLM call at each hop, so it suits a small number of well-structured documents. A vector database is the opposite trade: one millisecond ANN lookup that scales to millions of chunks but is flat and blind to document structure. Use vectors for large, messy corpora and speed; use PageIndex for bounded structured docs where the answer is found by reasoning about where it lives; combine them by shortlisting with vectors then navigating within a document.
A vector database stores dense numerical embeddings alongside their source documents and uses approximate nearest-neighbor (ANN) algorithms to find the most semantically similar entries for a query vector in milliseconds. Unlike a keyword index, similarity is measured in geometric space so synonyms and paraphrases match naturally. Common choices include Pinecone, Weaviate, Qdrant, and pgvector for Postgres.
RAG augments an LLM by retrieving relevant documents from an external knowledge store at query time and feeding them into the prompt as grounding context. A basic pipeline chunks and embeds documents into a vector store, retrieves the top-k most similar chunks for a query, and the LLM generates an answer conditioned on them, reducing hallucination and keeping knowledge current.
Retrieval cheaply searches a large corpus and returns a candidate set, prioritizing recall. Reranking applies a more expensive query-document model to that small set and improves precision and ordering at the top. A reranker cannot recover relevant documents absent from the retrieved candidates, so evaluate first-stage recall separately.