Anthropic's Claude Fable 5.2 Tops Cognition's Coding Benchmark at a Third of the Cost

Cognition has added Anthropic’s Claude Fable 5.2 to Devin Desktop and the Devin CLI. The company’s launch data places the model first on its FrontierCode 2.0 coding-agent benchmark and shows its strongest efficiency gain at low reasoning effort.

The score comes with a cost advantage

At low reasoning effort, a setting that controls how much inference-time computation the agent uses, Fable 5.2 surpassed Fable 5.1’s best result at less than one-third of the older release’s cost per task, Cognition reports. The comparison applies to this benchmark and configuration; production costs will vary with task mix, context size, retries, and review outcomes.

For teams running many Devin jobs, cost per accepted patch provides the useful production measure because it combines model spend with retries, regressions, and engineering review time. Cognition’s result makes Fable 5.2 a candidate for a repository-specific evaluation at low effort.

Mergeability raises the bar

FrontierCode measures whether a coding agent can produce a patch that maintainers could merge. More than 20 open-source maintainers created realistic tasks from repositories they manage, with over 40 hours of work going into each task. The grading criteria cover correctness, design quality, and requirements inferred from the surrounding codebase.

Compared with FrontierCode 1.1, version 2.0 expands the workload and tightens the acceptance criteria:

• Broader source material: Tasks come from 55% more repositories, with security patches and performance optimizations added to the mix.
• Stricter grading: The evaluation includes 43% more blocking criteria, meaning requirements each patch must satisfy.
• Larger expected changes: Reference solutions touch 67% more files and 89% more lines of code on average.
• Bigger codebases: The evaluated repositories are 25% larger on average.

Cognition describes the benchmark as unsaturated, meaning scores remain far below the maximum and still separate models across capability levels. The company reports a wider score spread than on DeepSWE v1.1 and FrontierCode 1.1. Opus 5.5 previously led FrontierCode 2.0 at 45.5%; Fable 5.2 raises the reported best result to 49.1%.

Implicit requirements expose weak spots

Cognition uses the term thoroughness for requirements an agent should infer from the prompt and existing code, even when the instructions leave them unstated. Typical examples include:

• Fixing every occurrence of a defect when the prompt identifies only one instance.
• Updating sibling code paths that implement the same behavior.
• Extending support to adjacent input types already handled by comparable code.

FrontierCode 2.0 contains three times as many thoroughness criteria as version 1.1, and those checks account for a larger share of its blocking requirements. Cognition identifies thoroughness as the most common failure category among the evaluated frontier models. A patch can satisfy the named case and pass relevant tests while remaining unsuitable for merge because related paths, interfaces, or edge cases were left inconsistent.

More reasoning can backfire

Reasoning effort gives a model more computation for planning and checking its work. FrontierCode sometimes records lower scores at higher settings because agents tend to expand the scope of a patch, touch more files, and introduce changes that the task does not require. Additional correct code does not automatically improve the score, and unnecessary edits create more opportunities for regressions.

At higher effort, Cognition also observed more runs in which agents accidentally retrieved the solution commit, contaminating the evaluation. Other runs failed after the model developed an incorrect interpretation of the requested change and implemented it across a broader area of the codebase.

A PostHog task in the benchmark write-up illustrates the pattern. At low effort, GPT-6 Astra produced a small patch that passed the task. At maximum effort, it deleted a public API class and introduced a breaking change.

How to evaluate it in Devin

Teams testing Fable 5.2 can begin with low reasoning effort on representative repository tasks, then increase the setting selectively when their own results justify the added cost and broader changes. The benchmark suggests several useful test categories:

1. Multi-file refactors in large codebases where related paths must remain consistent.
2. Bug fixes involving defects repeated across sibling modules or implementations.
3. High-volume agent workloads where per-task cost and retry rates affect the budget.
4. Security and performance work, both of which are represented in FrontierCode 2.0.

A production pilot should track task completion, reviewer time, regressions, retries, elapsed time, and total cost per accepted patch. Comparing those measures across effort settings will show whether the benchmark’s cost advantage carries over to a team’s repositories and development practices.

Cognition created the benchmark and published the results, so the figures do not provide independent validation. The leading score remains below 50%, and thoroughness failures continue to block many patches. Repository-specific evaluations, automated tests, code review, and security checks remain necessary for production use.