Meta's FRESCO Catches AI Search Picking Outdated Facts 98% of the Time
Meta and UCLA researchers show that 19 popular re-rankers stubbornly prefer old, detailed documents over newer ones, and offer a fix without retraining.
ProTakeaways
- New benchmark FRESCO tests whether RAG re-rankers pick chronologically valid evidence, not just semantically similar text.
- Built automatically from Wikidata facts aligned with Wikipedia revision history, 7,316 queries with 51 candidates each.
- All 19 re-rankers tested show 84 to 98 percent Obsolete Ratio, preferring older detailed passages over newer correct ones.
- Qwen3-Reranker-8B leads overall but drops 8 MAP points when recency is phrased implicitly instead of explicitly.
- Pareto-based instruction search with textual gradients yields up to 27 percent gain without fine-tuning any weights.
- Code available at github.com/facebookresearch/fresco, method extends to RankGPT on LLaMA 3.2 as well.
Retrieval-augmented generation is only as good as the documents that reach the language model, and the re-ranker decides which candidates survive. A new benchmark from Meta Superintelligence Labs and UCLA called FRESCO exposes a serious blind spot in that gatekeeper: when facts change over time, today's re-rankers systematically pick the older, more detailed passage over the newer, correct one.
FRESCO (Factual Recency and Evolving Semantic COnflict) pairs recency-seeking queries with historical Wikipedia revisions to test whether re-rankers can prioritize factually current evidence while maintaining semantic relevance. The paper also introduces an instruction-only optimization method that recovers up to 27% on time-sensitive queries without touching model weights.
The problem hiding in every RAG stack
Most retrieval benchmarks like MS MARCO, TREC DL, Natural Questions, and BEIR treat relevance as a fixed property of a document. In production RAG, that assumption breaks constantly. A query asking for the latest stable version of vLLM might retrieve two changelogs, one reporting version 0.5.0 from weeks ago and another reporting 0.6.0 from two days ago. Both look highly relevant under semantic matching, but only the latter is chronologically valid.
The authors call this Evolving Semantic Conflict: multiple semantically relevant candidates whose content disagrees because the world moved on. Existing benchmarks give practitioners no visibility into whether their re-ranker handles it.
Mining Wikipedia's edit history
FRESCO's construction pipeline is clever because it needs no manual annotation. It mines Wikidata for relations with explicit temporal qualifiers such as start time or end time, then aligns each fact with the Wikipedia page revisions that existed during its validity interval. For a subject-relation pair like (Cristiano Ronaldo, member of sports team), the object shifts from Juventus to Al Nassr with corresponding validity periods, and both the current and prior states must be textually supported in Wikipedia.