---
title: "Johns Hopkins' DAR Framework Lets AI Look up Laws to Beat Legal Reasoning"
subtitle: "DAR gives LLMs on-demand access to legal statutes as tools, pushing state-of-the-art on rule-based reasoning — but only for the strongest models"
url: "https://alphasignal.ai/news/johns-hopkins-dar-framework-lets-ai-look-up-laws-to-beat-legal-reasoning"
author: "AlphaSignal Newsroom"
author_url: "https://alphasignal.ai/editorial-team"
company: "AlphaSignal"
company_url: "https://alphasignal.ai/company/alphasignal"
published: "2026-06-03T00:00:00.000Z"
updated: "2026-06-11T03:49:58.834Z"
type: "Paper"
topics:
  - "Agents"
  - "Reasoning"
subtopics:
  - "Agent Frameworks"
  - "Tool Use"
  - "Test Time Compute"
upvotes: 2663
source_url: "https://arxiv.org/abs/2606.05009"
comments: 0
truncated: true
---

# Johns Hopkins' DAR Framework Lets AI Look up Laws to Beat Legal Reasoning

> DAR gives LLMs on-demand access to legal statutes as tools, pushing state-of-the-art on rule-based reasoning — but only for the strongest models

Legal and policy reasoning is one of the hardest tasks you can hand an LLM. Not because the language is dense, but because the rules are long, cross-referenced, and change meaning depending on which clause you read first. A new paper from Johns Hopkins University and Télécom Paris tackles this head-on with [DAR: Deontic Reasoning with Agentic Harnesses](https://arxiv.org/abs/2606.05009), introducing an agentic framework that lets models look up statutes on demand rather than trying to hold an entire legal code in their context window.

## Why Legal Reasoning Breaks LLMs

**Deontic reasoning** is the formal term for applying explicit rules and obligations to specific facts to reach a conclusion. Think: computing a taxpayer's exact liability under the U.S. tax code, or determining whether an immigration appeal should succeed under USCIS administrative guidelines. It is not fuzzy or probabilistic. There is a correct answer derivable from the rules, and the model either gets it or it doesn't.

A key technical challenge for LLM-based deontic reasoning is that the relevant ruleset can be long and cross-referenced, so models may still fail to locate the rules needed for a particular reasoning step. Stuffing an entire statute into a context window is one approach, but it creates a needle-in-a-haystack problem: the model must still identify which specific sub-clause governs the current step of its reasoning, and frontier models routinely miss these lookups even when the text is right in front of them.

Across frontier LLMs and coding models, best hard-subset performance reaches only 44.4% on SARA Numeric and 46.6 macro-F1 on Housing , numbers that make clear this is not a solved problem, even for the most capable models available today.

## The Benchmark: DeonticBench

DAR is evaluated on **DeonticBench**, a multi-domain benchmark purpose-built for rule-based reasoning. DeonticBench covers four real-world rule domains: U.S. federal taxes (SARA), airline baggage policies, U.S. state housing law, and U.S. immigration administration (USCIS-AAO). All tasks follow a unified input-output format, but the domains vary in answer type and reasoning demands, including numeric computation, logical entailment, and multi-factor legal decision-making.

_Preview only. Full text requires an AlphaSignal Pro account._

## Sources

- [DAR: Deontic Reasoning with Agentic Harnesses](https://arxiv.org/abs/2606.05009)
- [DeonticBench: A Benchmark for Reasoning over Rules](https://arxiv.org/pdf/2604.04443)
- [LegalBench: A Collaboratively Built Benchmark for Measuring Legal Reasoning in Large Language Models](https://arxiv.org/abs/2308.11462)
- [Language Models and Logic Programs for Trustworthy Tax Reasoning](https://arxiv.org/html/2508.21051v3)
