EMERGE Benchmark Catches AI Failing to Keep Knowledge Graphs Current
A new benchmark pairs 233K Wikipedia passages with 1.45 million Wikidata edits to test whether models can actually keep knowledge graphs current.
- EMERGE pairs 233K Wikipedia passages with 1.45M Wikidata edits across seven yearly snapshots (2019-2025).
- Introduces five Text-driven KG Updating operations: Exists, Add, Mint+Add, Infer, and Deprecate.
- Accepted at ICLR 2026; benchmarks 13 models including GPT-5.1 with the EDC+ pipeline.
- Model performance degrades as KG deltas grow, revealing weakness on stale graphs.
- Deprecate and Infer operations are hardest, exposing lack of graph-aware reasoning in LLMs.
- Code at github.com/klimzaporojets/emerge, dataset on Hugging Face.
Knowledge graphs drift out of date as organizations merge, people change jobs, new entities appear, and old relationships expire. Information extraction systems usually read text and emit standalone triples without consulting the graph already in storage. EMERGE, a benchmark accepted at ICLR 2026, evaluates the harder task of updating an existing graph from new text.
Researchers from Aarhus University, the University of Amsterdam, and Sapienza University of Rome frame the problem as Text-driven Knowledge Graph Updating, or TKGU. Each update depends on two sources: the claims expressed in a passage and the current state of the knowledge graph. A correct system may preserve an existing edge, add one, create an entity, infer related edges, or retire an obsolete fact.
Five Ways a Graph Can Change
Conventional information extraction can add the triple “person, holds position, CEO” after reading an appointment announcement. A production knowledge graph also needs to resolve the person and company to existing nodes, retire the previous CEO relationship, and avoid duplicating facts already stored.
This story is for Pro members
You've reached the end of the free preview. Upgrade to AlphaSignal Pro to read the full article - and everything else behind the paywall.
Comments
