Cornell's DecisionRAG Beats Dense Retrieval by 17 Points Without Training
DecisionRAG replaces vector search with an LLM that judges candidates in parallel, beating dense retrieval plus an 8B reranker on multiple benchmarks.
- Cornell team proposes DecisionRAG, a retrieval method that uses an LLM instead of a vector index.
- Beats dense retrieval plus an 8B reranker by 13.8 R@1 on MultiDoc2Dial, 17.4 Hit@1 on Natural Questions.
- Scores all candidates at a hierarchy level in one forward pass with no token generation.
- Three stages: binary document judgments, passage judgments, then multiple-choice reranking.
- Latency stays at 0.7 to 1.8 seconds per query over 2.68 million Wikipedia paragraphs.
- No task-specific training and no first-stage retriever required; no code release announced.
In retrieval-augmented generation, the retrieval stage often sets the accuracy ceiling because it selects the evidence available to the generator. When the relevant passage never enters the candidate set, the generator lacks the material needed to answer accurately.
Researchers at Cornell propose using a pretrained language model to navigate a corpus without a separate lexical or dense first-stage retriever. Their method, DecisionRAG, reports double-digit top-rank gains over dense retrieval with an 8-billion-parameter reranker, with per-query latency of 0.7 to 1.8 seconds across 2.68 million Wikipedia paragraphs.
Why similarity drops evidence
BM25 depends on lexical overlap and can miss paraphrases, while dense retrievers compress each passage into a fixed vector whose representation cannot adapt to the query. Cross-encoder rerankers apply deeper reasoning, but only to candidates supplied by the initial retrieval stage.
DecisionRAG uses a pretrained language model as the relevance judge throughout retrieval. The model evaluates whether each candidate could answer the query, allowing its language understanding to guide pruning without task-specific fine-tuning.
A tree narrows the search
Before query serving, the corpus is organized into a hierarchy whose upper levels may contain topic clusters, followed by documents and passages. Native document structure can provide this hierarchy, while flatter collections require clustering or another preprocessing method.
At query time, the model evaluates the children of each retained node, keeps the highest-scoring candidates, and leaves pruned subtrees unexplored. The pipeline proceeds through three narrowing steps:
This story is for Pro members
You've reached the end of the free preview. Upgrade to AlphaSignal Pro to read the full article - and everything else behind the paywall.
Comments
