DeepSeek V4 Pro Hits 90% on ARC-AGI-1 but Stalls on Harder Puzzles
DeepSeek V4 Pro hits 90.5% on ARC-AGI-1 and 61.3% on ARC-AGI-2, but extra reasoning barely moves the needle on abstract puzzles.
- DeepSeek V4 Pro 0813 hits 90.5% on ARC-AGI-1 at $0.18 per task (low reasoning).
- ARC-AGI-2 tops out at 61.3% for $0.60 per task at max reasoning effort.
- Scores are essentially flat across low, high, and max reasoning variants.
- Without any reasoning, the model collapses to 13.0% / 0.8%, showing scaffolding drives performance.
- High reasoning used more tokens than max and beat it by 1-4 points due to timeouts.
- Top scores match the smaller DeepSeek V4 Flash, suggesting scale is not the bottleneck.
ARC Prize just published verified results for DeepSeek V4 Pro 0813 on the ARC-AGI benchmarks, and the numbers say something interesting about where reasoning models hit a wall. The model posts strong scores on the easier ARC-AGI-1 test but stalls on ARC-AGI-2, and cranking up its thinking budget barely helps.
According to the official results page, at max effort V4 Pro scores 90.0% on ARC-AGI-1 Semi-Private at $0.30 per task and 61.3% on ARC-AGI-2 Semi-Private at $0.60 per task. The low-reasoning variant actually edges out max on ARC-AGI-1, hitting 90.5% at just $0.18 per task. DeepSeek is delivering competitive puzzle-solving without burning through reasoning tokens.
Scoreboard across reasoning levels
ARC Prize tests each model at four settings: none, low, high, and max. Here is how V4 Pro breaks down:
| Variant | ARC-AGI-1 | ARC-AGI-2 |
|---|---|---|
| Max | 90.0% | 61.3% |
| High | 87.2% | 59.7% |
| Low | 90.5% | 56.3% |
| None | 13.0% | 0.8% |
The collapse from 90% to 13% when reasoning is disabled is the clearest signal in the data. The base model has almost no innate abstract pattern-matching ability, and essentially all of the performance comes from the chain-of-thought scaffolding on top.
A jagged curve where a smooth one belongs
Normally you expect more reasoning tokens to produce monotonically better scores. That is not what happened here. ARC Prize noted on X that V4 Pro used more reasoning tokens at high than at max, with high beating max by 1% on ARC-AGI-1 and 4% on ARC-AGI-2. Repeated timeouts also lowered recorded costs unevenly across reasoning levels, which contributes to the noisy trend.
DeepSeek's top scores also come in roughly on par with the smaller V4 Flash sibling. Paying more for the bigger Pro model does not buy a meaningfully better ARC-AGI result, which suggests the ceiling is set by the reasoning strategy rather than raw parameter count.
What ARC-AGI actually tests
ARC-AGI tasks are small colored grids where the model sees a handful of input-output examples, infers the transformation rule, then applies it to a new input. ARC-AGI-1 problems are solvable by most humans in a few seconds. ARC-AGI-2 was designed to be harder, requiring composition of multiple rules or tracking abstract properties like symmetry, counting, or object identity. The 30-point gap between V4 Pro's scores on the two benchmarks is typical of current frontier models, all of which struggle when the puzzle demands more than one abstraction step.
Reading the competitive picture
A few practical takeaways for anyone tracking reasoning-model economics:
- Cost efficiency: at $0.18 per task on ARC-AGI-1, low-reasoning V4 Pro is one of the cheaper ways to reach the 90% band on the easier benchmark.
- Diminishing returns: turning the reasoning dial from low to max adds only a few percentage points on ARC-AGI-2 and can hurt on ARC-AGI-1.
- No lift from scale: V4 Pro matching V4 Flash suggests that for grid-abstraction tasks, DeepSeek's post-training recipe matters more than model size.
- Reproducible: the runs can be replicated using the open benchmarking repo, and the model itself is available on Hugging Face.
ARC-AGI-3 numbers for V4 Pro are still pending. That benchmark tests agents in interactive novel environments rather than static grids, and ARC Prize has said those evaluations are more operationally intensive and will roll out over the coming weeks. If the pattern from the grid results holds, the reasoning-level curves will look just as noisy once agentic behavior enters the mix.
Comments
