Google's Gemini 4 Carbon Tops AI Rankings and Hunts Security Bugs for Days
Google's new frontier model targets autonomous cybersecurity defense and multi-day agentic coding, with early access limited to vetted defenders through the Fairwind program.
- Google announced Gemini 4 Carbon, its new frontier model focused on cybersecurity and long-horizon agentic coding.
- Ranks #1 on Artificial Analysis Intelligence Index v5.0 at 59, hitting 88.2% on SEC-bench Pro.
- Scores 54.8% on ExploitGym and 71.0% on CWE-bench for vulnerability validation and patching.
- Access limited initially to cyber defenders via the Fairwind Program, with wider rollout promised later.
- Introductory pricing matches Argon: $2 per million input tokens, $10 per million output tokens.
- Reports 0.0% success on Gray Swan prompt injection tests and ~80% drop in misalignment alerts.
Google has announced Gemini 4 Carbon, a frontier model designed for autonomous cybersecurity defense and software engineering tasks that can run for days. Carbon follows Gemini 4 Argon by two weeks, giving Google an unusually short interval between flagship model releases.
Access begins with a limited group of cyber defenders, followed by additional trusted partners through Google’s Fairwind Program. Google plans broader developer and business access later, but it has not announced general-availability dates for the Gemini API or Vertex AI.
Security benchmarks lead the pitch
Google reports that Carbon ranks first on version 5.0 of the Artificial Analysis Intelligence Index with a score of 59. The launch materials also cite gains in vulnerability discovery, exploit validation, patch generation, and web development.
| Evaluation | Result | What it measures |
|---|---|---|
| Artificial Analysis Intelligence Index v5.0 | 59, ranked first | Broad model performance across several evaluations |
| SEC-bench Pro | 88.2% | Discovery of complex vulnerabilities in real software |
| ExploitGym | 54.8% | Validation that a reported vulnerability can be exploited |
| Collinear CWE-bench | 71.0% | Generation of working patches for validated findings |
| WebDev Arena | 1,769 Elo, ranked fourth | Relative performance on full-stack web development tasks |
Carbon’s WebDev Arena score rose by 89 Elo points from the previous Gemini release. Elo measures relative performance, so rankings can change as competitors, votes, and evaluation conditions change.
Days-long tasks test agent stability
Google says its internal teams ran Carbon for days while the model investigated and patched subtle bugs across large codebases and low-level systems. Those workloads test whether an agent can preserve state, recover from failed approaches, and continue working without inventing progress or bypassing the intended task.
The announcement does not disclose the number of long-running tasks, their completion rate, the tools provided to the model, or the amount of human intervention. Those details will determine whether developers can reproduce the reported behavior in production agent systems.
| Measure | Reported change | Practical meaning |
|---|---|---|
| Terminal-Bench 4.0 cheating | 0.0% successful; 0.4% attempted | The model rarely tried to bypass the intended task, and no attempt succeeded in the evaluation. |
| High-severity misalignment alerts | Nearly 80% lower | Google observed fewer serious policy or goal-conflict alerts in internal coding environments. |
| Unsupported claims | About half as many as Argon | The model more often documented missing evidence or blocked actions instead of asserting unverified results. |
| Overreach | 26% lower | The model was less likely to act beyond the requested scope. |
| Under-service | 29% lower | The model was less likely to stop before completing permitted work. |
| Indirect prompt injection | 0.0% attack success | No evaluated attack in Gray Swan’s benchmark redirected the model through malicious external instructions. |
Several results come from Google’s internal environments or use relative changes without absolute baseline rates. Independent testing will be needed to establish how Carbon behaves with private repositories, custom tools, long context histories, and production access controls.
Defense shapes the rollout
Carbon covers three stages of vulnerability management: finding a flaw, confirming whether it can be exploited, and generating a patch. Connecting those stages could shorten remediation cycles, although each step requires review because false positives, incomplete exploit analysis, and unsafe patches can create additional risk.
Google’s Chrome team reports that Carbon produced more than 10 times as many high-quality patches as an unnamed large frontier model from another lab. The announcement does not provide the comparison model, sample size, acceptance criteria, or review process, which limits interpretation of that figure.
Google says Carbon is participating in the U.S. government’s voluntary process for pre-release model testing. The Fairwind Program will provide staged access to selected defenders before a broader release, allowing Google to monitor offensive misuse and collect feedback from security practitioners.
API pricing matches Argon
Google plans to launch Carbon at Argon’s introductory API rates: $2 per million input tokens and $10 per million output tokens. Cached input receives a 95% discount, reducing the introductory rate to $0.10 per million cached tokens.
| Token type | Introductory rate | Later rate |
|---|---|---|
| Input | $2 per million tokens | $4 per million tokens |
| Cached input | $0.10 per million tokens | $0.20 per million tokens if the 95% discount continues |
| Output | $10 per million tokens | $20 per million tokens |
Cached pricing can reduce costs for agents that repeatedly reuse system prompts, repository maps, dependency information, or other stable context. Google has not specified how long the introductory window will last or confirmed whether the same cache discount will apply after prices double.
Production tests remain
Carbon’s reported results focus on three persistent agent problems: maintaining progress across long tasks, reporting uncertainty accurately, and resisting instructions embedded in untrusted content. Developers evaluating the model will need evidence from their own repositories and toolchains before assigning it unattended security or deployment work.
- Reproduce benchmark results with documented tool access, token budgets, time limits, and pass criteria.
- Measure recovery from failed commands, stale state, interrupted sessions, and partial patches.
- Test prompt injection through issues, documentation, source comments, web pages, and tool output.
- Review generated exploits and patches inside isolated environments with approval gates.
- Confirm context limits, rate limits, latency, regional availability, logging, retention, and service-level terms.
- Track the Gemini API and Vertex AI release dates, which Google has yet to announce.
Fairwind partners will provide the first external signal on whether Carbon’s benchmark gains transfer to operational security work. Broader judgment will depend on reproducible evaluations, transparent deployment controls, and performance under the messy conditions of real codebases.
Comments
