Cognition's Devin Adds Gemini 4 Carbon, Hitting 40% Mergeability Score

Cognition has added Google’s Gemini 4 Carbon to Devin Desktop and Devin CLI, according to its launch post. The model is available now and scored 40.0% on Cognition’s mergeable pull-request benchmark, up from Gemini 4 Argon’s 26.3%, while using less compute.

A benchmark built for merge decisions

FrontierCode 2.0 evaluates whether an AI-generated pull request belongs in a production codebase. Reviewers consider requirements, implementation quality, scope, and engineering judgment. Version 2.0 asks whether a pull request should be merged; the earlier version asked whether a reviewer would merge it.

Open-source maintainers created the benchmark’s 132 private tasks from repositories they know, spending more than 40 hours on each task. The prompts deliberately leave some requirements unstated, forcing agents to inspect the codebase and infer an appropriate solution. Any submission that misses a blocking requirement receives zero, which keeps scores well below those on saturated coding benchmarks.

Carbon trails the benchmark leader, Claude Opus 5.5, by 5.5 percentage points. It leads GPT-6.1 Sol by one point and Argon by 13.7 points, making it Google’s strongest model in Cognition’s published results.

Cheaper runs, split token prices

Cognition reports that Carbon exceeds Argon’s best score at low reasoning effort, a setting that limits the inference compute allocated to a task. Carbon’s highest-scoring run also costs less than two-thirds as much per task as Argon’s strongest run.

Carbon’s published token prices vary by sales channel and pricing period. The introductory API rate matches Argon, while Google Cloud’s SKU catalog lists rates twice as high.

At $4 per million input tokens and $20 per million output tokens, Cognition says Carbon matches Claude Opus 5.5’s token rates after the introductory period. Actual task cost also depends on prompt size, generated output, tool calls, retries, and reasoning effort.

Google divides the workload

Google positions Carbon as the Gemini 4 option for demanding reasoning tasks, including architectural design, cross-file coordination, and code auditing. Argon remains the general-purpose choice for routine software engineering and knowledge work, where latency and token cost carry more weight.

Google’s launch data reports an 89-point Elo gain over Argon, placing Carbon at No. 4 on WebDev Arena. Elo is a relative ranking score derived from head-to-head comparisons. Google also says internal testing found roughly half as many unsupported claims, such as references to nonexistent APIs or invented function signatures. Both figures are vendor-reported and come from evaluations separate from FrontierCode.

Longer reasoning can widen the patch

Cognition found that higher reasoning effort sometimes prompted agents to modify code outside the requested scope. FrontierCode penalizes those additions because an otherwise functional patch can still create review risk, regressions, or unnecessary maintenance work.

A tightly scoped Devin task may therefore produce a more mergeable patch at low or medium reasoning effort. The benchmark’s private task set limits direct reproduction, so repository-level A/B tests remain necessary before adopting one setting across a team.

Routing Carbon in Devin

Devin users can divide work among models according to task complexity, scope, and review cost:

1. Assign Carbon to architectural changes, cross-file refactors, difficult debugging, and audit-style work.
2. Use Argon or lighter models for routine edits and rapid iteration in the Devin CLI.
3. Start narrowly scoped tasks at low or medium reasoning effort, then increase the budget when the model needs deeper analysis.
4. Require clarification when acceptance criteria are ambiguous. Cognition says Carbon is designed to ask questions instead of filling gaps with assumptions.
5. Run large jobs in isolated workspaces and use a separate verifier to inspect tests, scope, and unintended changes before merging.

Carbon usage flows through Devin’s existing model routing and billing. Teams comparing it with Claude can assign both models the same repository task, acceptance criteria, test suite, and review rubric, then measure mergeability, latency, and total cost.