Skip to main content
Back

UC Santa Barbara's CompliBench Reveals Frontier AI Judges Miss 70% of Agent Violations

A new benchmark from UCSB and Cresta reveals that even frontier LLMs miss guideline violations in enterprise dialogues, while a small fine-tuned judge beats them all.

Paper, Full Access
  • UCSB and Cresta release CompliBench, a benchmark for LLM judges on multi-turn compliance violation detection.
  • Pipeline uses adversarial flaw injection to synthesize challenging, precisely labeled violations across Airline, Healthcare, and Insurance domains.
  • 318 dialogues averaging 16.6 turns and 3.7 embedded violations, with turn-level ground truth labels.
  • Frontier models struggle: GPT-5 hits only 28 to 47 percent Conversation-Level Accuracy; Gemini-3-pro leads at 49 to 57 percent.
  • Fine-tuned Qwen3-8B beats GPT-5 across all domains and generalizes to unseen domains from Airline-only training data.
  • Code and data open sourced at github.com/UCSB-NLP-Chang/CompliBench.

Enterprise chatbots living inside airline call centers, insurance hotlines, and healthcare intake lines are supposed to follow strict operational playbooks: verify identity in a specific order, transfer certain requests to humans, never invent legal advice. How do you actually check whether an AI agent is following those rules across a 20-turn conversation? The usual answer, LLM-as-a-Judge, turns out to be shakier than anyone assumed. A team from UC Santa Barbara and Cresta has built a benchmark called CompliBench to measure how badly current judges fail at this job, and their results are unflattering across the board.

The gap nobody was measuring

Prior benchmarks like tau-bench and tau2-bench evaluate task-oriented agents mostly on whether the final state is correct: was the refund issued, was the flight rebooked. Existing evaluations are largely limited to single-turn or weakly contextual settings and do not explicitly consider task-oriented dialogue systems, which require maintaining long-range dependencies and adhering to complex guidelines across multiple turns. An agent can technically complete a task while breaking half a dozen rules along the way, and nobody notices because the final answer looked right.

Building a benchmark for turn-level compliance is hard for two reasons the authors flag directly. First, annotating real conversations with the exact turn where the exact guideline was breached is prohibitively expensive and hits privacy walls. Second, no one had a reliable way to synthesize realistic agent violations, because a naive LLM asked to "generate a violation" usually produces something either laughably obvious or indistinguishable from compliant behavior.