STAIR Beats Every RAG Baseline by Reading Documents Like a Table of Contents
A new generative retriever leverages the table of contents to hit 82.6% Recall@1 on a fresh 18-book benchmark, with hallucinations under 0.05%.
PRO- STAIR is a generative retriever that uses a document's table of contents as the retrieval target instead of chunk embeddings.
- Hits 82.6% Recall@1 on the new SearchTome benchmark, beating DSI (76.9%), DPR (68.7%) and BM25 (59.5%).
- Reports a hallucination rate under 0.05% by constraining outputs to valid ToC paths.
- SearchTome ships 18 books across 6 domains with train, dev, test queries and gold ToC labels.
- Built on Differentiable Search Index; the model generates section paths directly from queries.
- Code and data available via an anonymous repository linked from the arXiv paper.
Most retrieval augmented generation pipelines treat a document like a stack of Lego bricks. They slice it into fixed-length chunks, embed each one, and hope cosine similarity does the rest. In the process, the map of the document, its chapters, sections, and subsections, gets thrown out. A new paper argues that map is exactly what a retriever should be reading.
The system is called STAIR (STructure Aware Information Retriever), and it comes paired with a benchmark called SearchTome. STAIR hits Recall@1 of 82.6% on SearchTome versus 76.9% for DSI, and beats other strong baselines including BM25 (59.5%), DPR (68.7%), and out-of-the-box Mistral (13.8%). The gap over DSI is reported as statistically significant, and the hallucination rate sits below 0.05 percent.
Chunking throws away the map
LLMs keep getting better at long context, but they still suffer from the "lost in the middle" problem, where facts buried deep in a huge prompt get ignored. Precise retrieval remains a hard requirement. The standard recipe of splitting a book into 512-token windows discards the semantic hierarchy that the author put there on purpose.
Think about how a human answers a question from a textbook. You do not fuzzy-match paragraphs. You skim the table of contents, jump to the likely chapter, then narrow down. Length-based chunking flattens that hierarchy into a bag of passages. STAIR instead lets an LLM exploit the global structure, such as a Table of Contents, to store and retrieve information from its own weights.
This story is for Pro members
You've reached the end of the free preview. Upgrade to AlphaSignal Pro to read the full article - and everything else behind the paywall.
Comments
