What is vectorless retrieval (PageIndex), and when would you use it over a vector database?

How to think about it

Use PageIndex over a vector database when the document’s structure tells you where the answer is: a contract clause, a filing note, a chapter in a manual, or a particular appendix. It is not a faster replacement for vector search. It trades embedding-based lookup for LLM-guided navigation through a document tree, so it is most useful for a bounded set of long, well-organized documents.

Why this works

Standard vector retrieval turns each text chunk into an embedding, a list of numbers representing its meaning. The query is embedded in the same way. The system then finds chunks whose vectors are close to the query vector, often using cosine similarity, which measures the angle between two vectors. A vector database makes this search efficient with approximate nearest-neighbor search, or ANN: an index finds likely matches without comparing the query with every stored chunk.

That is excellent when the question is essentially, “Which passages talk about this idea?”

The weakness is that semantic closeness is not the same as document relevance. A chunk about “debt maturity,” “lease obligations,” and “liquidity risk” may be close to a question about a bond maturity schedule even when it comes from the wrong note. The vector search sees related language. It does not inherently know that Note 18 is the authoritative section and Note 7 is a distracting cousin.

When people call vector retrieval “flat,” they mean flat in its document semantics, not necessarily its physical implementation. An HNSW index, for example, is internally a graph. But the retrieval decision is still usually based on chunk similarity rather than “the financial statements chapter contains the borrowing note.”

PageIndex takes a different approach. It parses a document into a hierarchical tree: a root for the whole document, children for chapters or major sections, and deeper nodes for subsections. Each node typically carries a title, a short summary, and a page range. The large language model, or LLM, reads the question and the candidate nodes, chooses the most relevant branch, and continues downward until it reaches the likely evidence.

The summary is a routing map. It is not the evidence itself.

At the end, the system reads the actual text from the selected pages and gives that text to the model for the answer. That final distinction matters. A good implementation does not ask the LLM to answer from a vague generated summary when the source page contains the legally or financially important wording.

A concrete example

Imagine a 240-page bank annual report. It has a table of contents, twelve major sections, numbered notes, and page numbers that match the PDF.

The question is:

“How much of the company’s borrowings mature in 2028?”

A conventional vector pipeline might split the report into roughly 500 overlapping chunks, embed them, and retrieve the eight closest chunks. It could return passages from the borrowing note, the lease-liability note, the liquidity-risk discussion, and the accounting-policy section. All of them contain language about obligations or maturity. The answer may still be present, but the model has to sort out the duplicates and near-misses.

A PageIndex-style tree might look like this:

Annual report: pages 1-240
├── Business overview: pages 1-24
├── Risk factors: pages 25-51
├── Financial statements: pages 96-154
│   └── Note 18, borrowings: pages 132-139
└── Appendices: pages 220-240

The model can reason:

1. The question asks about a financial amount, so “Financial statements” is more likely than “Business overview.”
2. Within that branch, “Note 18, borrowings” is more relevant than a general risk discussion.
3. The answer should come from the maturity table on pages 136 and 137.

If the table shows $80 million in the 2028 column, the system can answer with that figure and cite the pages. The important improvement is not that the model magically understands finance. It is that the retrieval process uses the report’s own organization as a strong prior.

A three-level path means roughly three navigation decisions, usually involving model calls, before the final answer step. That is much more work than a database lookup, but it can prevent the classic failure where the system retrieves a plausible paragraph from the wrong part of a long document.

The trade-off

Dimension	Vector database	PageIndex-style retrieval
Representation	Embeddings for chunks	Hierarchical document tree
Main query operation	Similarity search with ANN	LLM-guided branch selection
Best scale	Large, growing corpora	Bounded sets of structured documents
Main strength	Speed and broad semantic recall	Structure, page awareness, explainability
Main weakness	Can ignore document hierarchy	Can choose the wrong branch and costs more per query

The latency comparison needs care. A tuned vector lookup may take only milliseconds, but the complete vector pipeline still includes query embedding, filtering, reranking, and answer generation. PageIndex adds model calls during retrieval, so its latency and token cost generally increase with tree depth. The exact numbers depend on the model, context size, service, and whether intermediate results are cached.

When I would choose PageIndex

I would choose it when all three of these conditions are mostly true.

First, the corpus is bounded. This might be one regulatory filing, a collection of 50 product manuals, or the current versions of a few hundred contracts. The system does not need to search tens of millions of unrelated chunks on every request.

Second, the documents have meaningful structure. Headings, numbered clauses, chapters, tables of contents, page ranges, and appendices give the tree useful information. A question such as “What are the termination rights?” is easier to route in a contract whose sections are named “Term and Termination” than in a bag of chat messages.

Third, the location of the answer matters. Legal, compliance, financial, and technical questions often need the exact section and page, not merely a textually similar passage. A citation to “Note 18, pages 136–137” is easier for a reviewer to audit than eight semantically related chunks with no coherent document path.

PageIndex is particularly attractive when explainability means showing the route: whole document, financial statements, borrowing note, maturity table. That route is not a proof of correctness, but it gives an engineer or auditor something concrete to inspect.

When I would not choose it

I would not make PageIndex the sole retriever for a large, messy, high-throughput corpus. Consider a support platform with 30 million tickets, product pages, emails, and chat transcripts. Most documents have weak hierarchy, many are short, and users expect low latency. Precomputed embeddings, metadata filters, keyword search, and reranking are a better fit because ANN can narrow a huge search space without navigating thousands of individual document trees.

I would also avoid relying on it when questions routinely span many documents. PageIndex is naturally document-centric. It can find the right section inside a chosen report, but something still has to choose which reports to inspect.

There is another trap: “vectorless” does not mean “preprocessing-free.” The system still needs PDF parsing, heading detection, page alignment, and often OCR. A scanned 400-page contract with crooked pages and broken tables may produce a beautifully shaped tree whose leaves contain damaged text. The tree cannot repair bad extraction by itself.

For documents with no reliable structure, keyword or hybrid retrieval may be safer. An exact identifier such as AB-2047, a product serial number, or a clause number is often better handled by lexical matching than by semantic similarity or LLM navigation.

The production pattern I would use

In production, I would usually compose the methods rather than declare one the winner.

At the corpus level, use metadata, keyword search, or vector retrieval to shortlist candidate documents. For the annual-report example, the query might first identify the relevant bank and reporting year. Then PageIndex navigates inside that report to the borrowing note.

The query path becomes:

1. Find likely documents.
2. Load the tree for each document version.
3. Navigate from the root to one or more relevant branches.
4. Read the source pages at the selected leaves.
5. Answer with page citations and retain the retrieved evidence for inspection.

The tree should be built once per document version and cached. It should not be regenerated for every question. I would also store the original file hash, parser version, tree artifact, and page ranges so a later answer can be traced to the exact source that produced it.

I would keep a fallback. If the model cannot confidently select a branch, search the document with a lexical or vector method instead of returning “not found.” For high-stakes material, retaining two plausible branches is often safer than committing early to one. That is a retrieval policy, not a magical PageIndex feature, and it should be evaluated explicitly.

Failure modes to watch for

The first symptom of a bad tree is often a confident answer from the wrong section. The node summaries may be too vague, the table of contents may have been parsed incorrectly, or a page range may point to the wrong PDF pages. Log the selected path and check whether the correct section was ever considered.

The first symptom of an OCR or table-extraction failure is a citation to the right page with the wrong number. Financial tables are especially unforgiving: a misplaced column can turn $80 million into $8 million without producing an obvious error message. Use layout-aware extraction, validate page alignment, and inspect the original page image for important numeric answers.

The first symptom of a scale mismatch is rising tail latency and token cost as documents become deeper. If the 95th-percentile response time grows from two seconds to nine seconds after adding longer manuals, the traversal depth and intermediate context are suspects. Cache trees, cap navigation work, route only into candidate documents, and fall back to vector or lexical retrieval where structure is not earning its keep.

What they’ll ask next

Is PageIndex really vectorless?
It is vectorless in the retrieval sense: it does not need to embed every chunk and perform nearest-neighbor search to locate evidence. It still uses models, parsing, summaries, and possibly ordinary keyword search. “No vectors” does not mean “no computation.”

Is PageIndex more accurate than vector search?
Not universally. It can improve structural retrieval for long, organized documents, but it can fail badly when the tree is wrong or the model prunes the correct branch. I would compare both systems on the same questions using evidence recall, answer correctness, citation or page accuracy, latency, and cost.

Would you combine PageIndex with a vector database?
Yes. Use vectors or hybrid search to shortlist documents at corpus scale, then use PageIndex to navigate within each promising document. That gives the vector system the job it is good at and gives the tree the job it is good at.

Say this in the interview: PageIndex uses an LLM to navigate a document hierarchy instead of retrieving embedding-nearest chunks, so I would use it for a small set of structured, page-sensitive documents and combine it with vector or hybrid search when the corpus is large.