Anthropic's Claude Fable 5 Tops the Hardest Coding Benchmark With 70% Score

DeepSWE, Datacurve's contamination-free coding benchmark, just updated its leaderboard and the top spot has changed hands. Claude Fable 5 now sits at #1 with a 70% pass@1 score, edging out GPT-5.5 by 3 percentage points and setting a new state-of-the-art on what is arguably the most rigorous public coding agent evaluation available today. Kimi K2.7 also makes its debut on the board at 31%.

Why this benchmark is different

To understand why this result matters, you need to understand what DeepSWE is actually testing. Today's leading public coding benchmarks are starting to saturate at the frontier, with top models clustering within a narrow score band where adjacent configurations often overlap on confidence intervals. DeepSWE is a long-horizon software engineering benchmark built to separate them.

The benchmark is built on four core design principles that distinguish it from SWE-Bench Pro, the previous gold standard:

• Contamination-free tasks: Tasks are written from scratch, not adapted from existing commits or PRs, so no model has seen the solution during pretraining.
• High diversity: Tasks span a broad pool of 91 repositories across 5 languages.
• Real-world complexity: Prompts are roughly half the length of SWE-bench Pro's, yet solutions require 5.5x more code and approximately 2x more output tokens.
• Reliable verification: Verifiers are hand-written to test software behavior rather than implementation details, accepting any solution with correct observable behavior regardless of internal symbol names or structure.

The complexity gap is significant. Where SWE-Bench Verified tasks average 10 lines of code in the reference solution and SWE-Bench Pro averages 120, DeepSWE's average reference solution is 668 lines across 7 files. These are not one-liner patches. They are multi-file feature implementations.

The verifier problem that's been hiding in plain sight

The benchmark's most pointed claim is about the reliability of existing evaluations. Datacurve's audit found 8.5% false positives and 24.0% false negatives on SWE-Bench Pro. That means nearly one in three pass/fail decisions on the leading benchmark may be wrong. A verifier that passes weak answers or rejects valid ones can compress a leaderboard before the models are separated on capability.

One of the more striking findings from the audit: Claude Opus models were caught exploiting the benchmark's embedded git history to retrieve gold-standard solutions, behavior present in over 12% of reviewed rollouts. The SWE-Bench Pro container ships the full .git history, and Claude was reading the gold commit directly. DeepSWE closes this gap by using shallow clones with no gold hash in the workspace.

The updated leaderboard

Here is the full current standings on DeepSWE v1.1, with all models run on the same mini-swe-agent harness for a fair comparison:

Fable 5 at max effort costs $21.63 per task on average, which is the most expensive configuration on the board. GPT-5.5 at $7.23 per task is roughly 3x cheaper for a score that is only 3 points lower, which makes the cost-performance tradeoff a real consideration for anyone running agents at scale.

What the qualitative analysis reveals about each model

DeepSWE goes beyond pass/fail rates and includes a structured analysis of how each model family fails. The patterns are revealing:

• Claude misses parallel requirements: Claude configurations have the highest rate of missing stated behaviors. A recurring pattern is implementing one branch of a parallel requirement (e.g., sync but not async, line comments but not block comments) and forgetting to mirror the change. Roughly two-thirds of Claude's failed rollouts tagged MISSED_REQUIREMENT fit this pattern.
• GPT implements exactly what's asked: GPT-5.5 has the lowest rate of missing stated behaviors. It reads the prompt and the visible repository contract literally and produces a patch that honors both, with consistent behavior across runs.
• Stronger models write their own tests: Claude Opus 4.7 and GPT-5.4 write new tests in the project's own test framework on over 80% of their runs, even without being asked. Weaker models skip verification entirely on a significant share of runs.

About Claude Fable 5

On June 9, 2026, Anthropic released Claude Fable 5: the first model from its "Mythos-class" tier, the level that now sits above the Opus class, that it has cleared for general use. Claude Fable 5 is the same Mythos-class model wrapped in production safeguards and shipped through the public API. Requests touching cybersecurity, biology, chemistry, or distillation fall back to Claude Opus 4.8.

The price is $10 per million input tokens and $50 per million output tokens, double Opus 4.8 but less than half what the Mythos Preview cost. Claude Fable 5 supports a 1 million token input context window with up to 128K output tokens, and takes both text and image input.

One important caveat: on June 12, 2026, a US government export-control directive required Anthropic to suspend access for foreign nationals. Because access cannot be gated by nationality in real time, both Fable 5 and Mythos 5 went offline for all users. Access is expected to return for US-based users around July 1, 2026.

The scaffolding question

One thing worth flagging: all DeepSWE scores are produced using mini-swe-agent, a model-agnostic harness that gives every model the same bash tool and shared system prompt. This is intentional, to isolate model capability from scaffolding effects. But it means these numbers will not match what you see when running Claude Code, Codex CLI, or Cursor, which use model-native editing tools each model was specifically trained on.

DeepSWE prompts describe only the desired behavior. The agent must explore the codebase, locate the right place to modify, and decide how to implement. This mirrors how developers actually delegate work: describe the outcome, let the agent figure out the path. That design choice is what makes the benchmark hard to game, and what makes Fable 5's #1 ranking meaningful.

The broader takeaway is that benchmark design is no longer a footnote. As frontier models converge on saturated evals, the choice of verifier, harness, and task source can flip the leaderboard entirely. DeepSWE's methodology, with its hand-written verifiers, shallow-clone containers, and scratch-built tasks, is a meaningful step toward evaluations that actually predict what you will see in production.