Datacurve's DeepSWE Shows Claude Opus 5 Beats Rivals at Half the Cost

Datacurve just added Claude Opus 5 to DeepSWE, its contamination-free long-horizon coding benchmark, and the results are notable: Opus 5 lands at 74% on v1.1, taking the top spot on the leaderboard. More interesting than the raw number is what it reveals about how Opus 5 behaves compared to its more expensive sibling, Claude Fable 5, and why this particular benchmark is worth paying attention to.

Why DeepSWE is a different kind of benchmark

Most public coding benchmarks are starting to saturate. On Scale AI's SWE-Bench Pro, top models cluster within a narrow score band, making it nearly impossible to tell which agent will actually perform best in a real codebase. OpenAI's GPT-5 family, Anthropic's Claude Opus, and Google's Gemini Pro have all converged within that narrow band.

DeepSWE is a 113-task evaluation spanning 91 open-source repositories and five programming languages that produces a dramatically wider spread among the same frontier models. Three design choices drive that separation:

• Contamination-free tasks: Tasks are written from scratch, not adapted from existing commits.
• Harder prompts, larger solutions: DeepSWE prompts are roughly half the length of SWE-bench Pro's, yet reference solutions average 668 lines of code added across 7 files, compared to SWE-bench Pro's 120 lines across 5 files.
• Reliable verification: v1.1 scores committed code in a clean, isolated environment. The verifier false positive rate is 0.3% versus SWE-Bench Pro's 8.5%, and the false negative rate is 1.1% versus 24.0%.

The full leaderboard

Here is every ranked model at max effort on DeepSWE v1.1:

The spread tells the real story. On SWE-Bench Pro, the gap between best and worst is around 30%. On DeepSWE, it reaches 70%. A harder benchmark surfaces differences that easier ones paper over.

Opus 5 vs. Fable 5: the quiet worker wins

The most practically useful finding is the behavioral contrast between Opus 5 and Fable 5 when running as agents. Opus 5 comments on just 11% of its tool calls; Fable 5 comments on 65%, narrating every step without ever producing a final summary. Opus works silently and delivers a summary at the end.

That behavioral difference compounds into a meaningful cost gap:

• Score: Opus 5 at 74% vs. Fable 5 at 70%
• Cost per task: $11.84 vs. $21.63, a 45% saving
• Token price: Opus 5 costs $5/$25 per million input/output tokens vs. Fable 5's $10/$50

Both models emit roughly the same number of output tokens per task (118k vs. 119k), so the cost gap comes almost entirely from Fable 5's higher per-token rate rather than any difference in verbosity at the token level.

Where Opus 5 falls short

Datacurve's 74% figure is their independent run. Anthropic's own system card reports 68.8% on DeepSWE v1.1, likely because Anthropic ran the model under its native Claude Code harness rather than the standardized mini-swe-agent DeepSWE uses for all models.

There are also domains where Opus 5 does not lead. On the Legal Agent Benchmark, Opus 5 scores 11.7% versus Fable 5's 13.3%. And the qualitative analysis from the original DeepSWE paper surfaces a recurring weakness across Claude models generally: they miss stated requirements more than any other model family. The pattern is consistent. When a prompt enumerates parallel behaviors, such as "support both sync and async," Claude tends to implement the obvious branch and skip mirroring its changes to the other. Roughly two-thirds of Claude's missed-requirement failures fit that shape.

What these tasks actually look like

DeepSWE tasks are not autocomplete exercises. Representative examples include:

• Adding deterministic map conflict detection to Y.Map writes in the Yjs CRDT library
• Fixing PromQL label sorting across typed and untyped values in Prometheus
• Adding trap coredump generation to the Wasmi WebAssembly engine in Rust
• Implementing XML diff, patch, and merge operations in the Go etree library

Each task requires the agent to explore an unfamiliar codebase, understand its architecture, implement a non-trivial feature across multiple files, and leave the existing test suite passing. That profile maps directly onto what Opus 5 is marketed for: complex, long-horizon agentic coding work.

How to run it yourself

DeepSWE is fully open. All 113 tasks, the evaluation harness, and every model trajectory are publicly available. You can run your own model or agent against the benchmark via the DeepSWE run page and browse the full task set and individual rollouts on the GitHub repo. Every model runs under mini-swe-agent with the same bash tool and shared system prompt, no per-vendor scaffolding.

One practical signal for teams building agentic coding pipelines: Anthropic removed more than 80% of Claude Code's system prompt for Opus 5 and Fable 5 with no measurable loss on coding evaluations. Both models follow instructions reliably enough that rules written to guard against older model behavior now cost quality rather than add it. Your prompt engineering overhead for Opus 5 should be lower than for previous generations.

At the top of the leaderboard, the margin between first and third place sits within the confidence intervals. What separates the contenders is cost efficiency and behavioral reliability across long, multi-step tasks, which is precisely what DeepSWE is designed to measure.