Skip to main content
Back

RegReAct Beats GPT-4o by 15 Points Parsing Unreadable EU Compliance Law

A seven-stage agent pipeline with self-correction loops turns dense EU sustainability regulations into machine-readable compliance criteria, beating GPT-4o across every metric.

Pro

  • RegReAct decomposes regulatory extraction into seven agents with Observe-Diagnose-Repair loops grounded in source text
  • Applied to three EU Taxonomy Delegated Acts, producing 242 activities and over 4,800 structured criteria
  • Beats GPT-4o single-pass by +15.52 on structural F1 and +13.1 on evaluation logic accuracy
  • Typed criterion graph enforces hierarchy well-formedness and cycle-free inheritance across six relationship types
  • Inline RAG embeds retrieved legal references directly into output, producing self-contained compliance records
  • Code, pipeline, and EU-TaxoStruct dataset released at github.com/RECOR-Benchmark/RECOR

Regulatory text is the kind of document that breaks language models. It nests conditions inside conditions, cross-references other regulations that reference yet more regulations, and buries thresholds in footnotes that determine whether a company is legally compliant. A new paper introduces RegReAct, a multi-agent framework that treats regulatory extraction less like a summarization problem and more like a compiler pass, with self-correcting stages that validate every output against the source.

The team applied it to three EU Taxonomy Delegated Acts, the sustainability regulation that determines which economic activities count as green under EU law, and built a structured dataset of 242 activities and over 4,800 criteria in the process.

Why single-pass LLMs choke on compliance text

Regulatory documents are unforgiving. The EU Taxonomy sets technical screening criteria that companies and investors use to classify whether an activity contributes to climate objectives, and stakeholders have repeatedly flagged implementation challenges tied to how complex those criteria are. Feed one of these documents into GPT-4o and ask for a JSON breakdown, and the output looks plausible but tends to invent structure that isn't there, drop nested conditions, and mangle cross-references.

The authors identify three specific failure modes: single-pass language models hallucinate structural elements, lose hierarchical relationships, and fail to resolve inter-document dependencies. That last one matters most in practice. A criterion in the EU Taxonomy might say "in accordance with section 5.11 of this Annex" or point to an ISO standard, and without resolving that pointer, the extracted rule is unusable for automated compliance checks.

The Observe-Diagnose-Repair loop

RegReAct's core mechanism is a self-correction loop attached to every stage of the pipeline. Each agent does more than produce output and pass it along. It observes its own output against the source document, diagnoses where things went wrong using a domain-specific issue taxonomy (structural, semantic, completeness, consistency), and