---
title: "DDOR Cuts AI Overrefusals by 86% Without Touching Model Weights"
subtitle: "DDOR borrows a classic software debugging algorithm to automatically find and fix the exact phrases that make LLMs refuse perfectly safe prompts"
url: "https://alphasignal.ai/news/ddor-cuts-ai-overrefusals-by-86-without-touching-model-weights"
author: "AlphaSignal Newsroom"
author_url: "https://alphasignal.ai/editorial-team"
company: "AlphaSignal"
company_url: "https://alphasignal.ai/company/alphasignal"
published: "2026-06-02T00:00:00.000Z"
updated: "2026-06-10T06:47:15.671Z"
type: "Paper"
topics:
  - "Security"
  - "Llms"
subtopics:
  - "Red Teaming"
  - "Jailbreaks"
upvotes: 1465
source_url: "https://arxiv.org/abs/2606.03601"
comments: 0
truncated: true
---

# DDOR Cuts AI Overrefusals by 86% Without Touching Model Weights

> DDOR borrows a classic software debugging algorithm to automatically find and fix the exact phrases that make LLMs refuse perfectly safe prompts

Every developer who has shipped an LLM-powered product has hit the same wall: a user asks something completely reasonable, and the model refuses. Ask about "how to kill a Python process" and some models will decline, pattern-matching on the word "kill" rather than understanding context. This is overrefusal -- and it turns out the research community has been borrowing a 20-year-old algorithm from software testing to finally crack it open.

[DDOR (Delta Debugging for OverRefusal)](https://arxiv.org/abs/2606.03601) is a new automated framework that does two things no prior tool could do together: it tells you **exactly which phrase** in a prompt is causing an unjustified refusal, and then it fixes it -- all without ever touching the model's weights or seeing inside its safety mechanisms.

## The Safety Tax Is Real

While significant research focuses on mitigating harmful content generation, enhanced safety often comes with the side effect of over-refusal, where LLMs may reject innocuous prompts and become less helpful. This is sometimes called the "safety tax" -- the cost in usability paid for every unit of safety gained. The problem is not hypothetical. A comprehensive study measured the over-refusal of 32 popular LLMs across 8 model families, and the results showed a strong correlation: the safer a model is at rejecting truly toxic prompts, the more likely it is to also reject harmless ones.

Existing benchmarks like [OR-Bench](https://arxiv.org/abs/2405.20947) have tried to quantify this. OR-Bench comprises 80,000 over-refusal prompts across 10 common rejection categories, a subset of around 1,000 hard prompts that are challenging even for state-of-the-art LLMs, and an additional 600 toxic prompts to prevent indiscriminate responses. But these benchmarks have a critical blind spot: a systematic measurement is challenging due to the difficulty of crafting prompts that can elicit the over-refusal behaviors of LLMs. They also offer no explanation for _why_ a refusal happens, and they are static -- they cannot adapt to the quirks of a specific model.

## Borrowing from the Bug-Hunt Playbook

The core insight behind DDOR is to treat overrefusal like a software bug. Delta Debugging is a widely used family of algorithms to automatically minimize bug-triggering test inputs. It takes a test input as a list of elements, partitions the list into sets at different granularities, and systematically identifies and deletes partitions that are bug-irrelevant. In classic software testing, you feed it a 10,000-line file that crashes a compiler, and it spits back the 3 lines that actually cause the crash.

_Preview only. Full text requires an AlphaSignal Pro account._

## Sources

- [DDOR: Delta Debugging for Explainable Overrefusal Testing and Repair (arXiv)](https://arxiv.org/abs/2606.03601)
- [Automated Overrefusal Prompt Generation and Repair with Delta Debugging (OpenReview / ICLR 2026 submission)](https://openreview.net/forum?id=Lx4MhiIzhH)
- [OR-Bench: An Over-Refusal Benchmark for Large Language Models](https://arxiv.org/abs/2405.20947)
- [WDD: Weighted Delta Debugging (arXiv)](https://arxiv.org/pdf/2411.19410)
