Google's Gemini 3.7 Flash Hits 84.6% on AGI Benchmark at 2.4x Lower Cost

Google's newest reasoning model has just delivered one of the sharpest cost-performance jumps we've seen on the ARC-AGI benchmark. At high effort, Gemini 3.7 Flash scores 95.5% on ARC-AGI-1 Semi-Private at $0.12 per task and 84.6% on ARC-AGI-2 Semi-Private at $0.25 per task. That 84.6% figure sits just a whisker below the 85% target the ARC Prize team set as the threshold that a solver would need to cross on ARC-AGI-2.

For context, ARC-AGI is the benchmark from François Chollet's team designed to measure fluid reasoning rather than memorized knowledge. ARC-AGI-2 is designed to stress-test the capabilities of state-of-the-art AI reasoning systems, provide useful signal on AGI progress, and inspire researchers to work on new ideas. The tasks demand things like symbolic interpretation, compositional reasoning, and rules that must be applied differently depending on context, all of which historically wrecked frontier LLMs.

The generational leap in one chart

The most striking thing about this result is how much better Gemini 3.7 Flash is than its predecessor released a few weeks earlier. At high effort, Gemini 3.6 Flash scores 91.2% on ARC-AGI-1 Semi-Private at $0.34 per task and 60.4% on ARC-AGI-2 Semi-Private at $0.61 per task. Compare that to the new numbers and the picture is unusual: accuracy went up while cost went down.

Benchmark	Gemini 3.6 Flash (High)	Gemini 3.7 Flash (High)
ARC-AGI-1	91.2% at $0.34	95.5% at $0.12
ARC-AGI-2	60.4% at $0.61	84.6% at $0.25

On ARC-AGI-2, that's roughly a 24 percentage point jump in accuracy paired with a 2.4x reduction in per-task cost. Very few generational releases move both axes in the right direction at once, which is why this result is drawing attention.

What the reasoning knobs actually do

Gemini 3.7 Flash ships with three reasoning effort settings, and the ARC results give a clean look at what each one buys you.

• High: 95.5% on ARC-AGI-1, 84.6% on ARC-AGI-2
• Medium: 91.2% on ARC-AGI-1, 63.7% on ARC-AGI-2
• Low: 85.2% on ARC-AGI-1, 52.9% on ARC-AGI-2

The gap between Medium and High on ARC-AGI-2 is worth staring at. Roughly 21 percentage points of accuracy live inside that knob, which means the model is doing a lot of its actual reasoning during extended thinking rather than in a single forward pass. If you're building on top of it, the effort setting is not a stylistic choice, it directly determines whether the model can attack novel puzzle-like problems at all.

Why the 85% number matters

The ARC Prize team explicitly frames 85% as the bar to clear on ARC-AGI-2. Can you create a system that can reach 85% accuracy? That threshold isn't arbitrary. 100% of tasks have been solved by at least 2 humans (many by more) in under 2 attempts. In other words, ARC-AGI-2 is calibrated so that the ceiling is human-solvable performance, not saturation from search or lookup.

Also worth flagging: cost is part of the benchmark now, not a footnote. Starting with ARC-AGI-2, all ARC-AGI reporting comes with an efficiency metric. We are started with cost because it is the most directly comparable between human and AI performance. The ARC team's philosophy is that a solver that needs unbounded compute to brute-force answers isn't demonstrating intelligence, it's demonstrating a big enough budget. That's why Gemini 3.7 Flash landing at $0.25 per task, rather than at the $5-plus per task numbers common among heavier reasoning systems, changes the story.

What ARC-AGI-2 actually tests

To understand why this jump matters, it helps to know what the benchmark is probing. ARC-AGI-2 focuses on three failure modes that tripped up earlier models:

1. Symbolic interpretation: Tasks requiring symbols to be interpreted as having meaning beyond their visual patterns. Current systems attempt to check symmetry, mirroring, and other transformations, and even recognize connecting elements, but fail to assign semantic significance to the symbols themselves.
2. Compositional reasoning: Tasks requiring simultaneous application of a rules, or application of multiples rules that interact with each other. In contrast, if a task has very few global rules, current systems can consitently discover and can apply them.
3. Contextual rule application: Tasks where rules must be applied differently based on context. Systems tend to fixate on superficial patterns rather than understanding the underlying selection principles.

These are exactly the categories where scaling alone historically hit a wall. ARC-AGI-1 was created in 2019 (before the rise of LLMs). It endured five years of global competitions, a 50,000x scale-up of base LLMs, and saw little progress until late 2024, with the introduction of test-time adaptation methods pioneered by ARC Prize 2024 entrants and OpenAI. Progress on ARC-AGI-2 has been driven by better inference-time reasoning, not bigger base models.

What this means if you're building with it

A few practical takeaways for anyone shipping products or research on top of Gemini's cheaper tier:

• The Flash tier is no longer just a latency-and-cost option. On abstract reasoning tasks it now trades punches with the heavyweight class, provided you turn reasoning effort up.
• Reasoning effort is the real dial. Medium to High on ARC-AGI-2 is the difference between 63.7% and 84.6%, so if your workload involves novel problem structure, budget for High.
• The cost-per-task math changes the economics of agentic loops. A 2.4x price drop combined with a big accuracy jump makes multi-step retry, self-critique, and verification loops meaningfully cheaper to run.

The ARC team also noted that ARC-AGI-3 evaluations for Gemini 3.7 Flash are still in progress, and that benchmark is a different beast. ARC-AGI has evolved from its first versions (ARC-AGI-1 and 2) which measured passive fluid intelligence, to ARC-AGI-3 which challenges AI agents to adapt on the fly to novel interactive environments. Whether Gemini 3.7 Flash can carry this efficiency curve into interactive agent settings is the next open question, and results should land in the coming weeks on the ARC Prize leaderboard.