KAIST's GRAM Beats 671B DeepSeek-R1 on Sudoku With a 10M Model
GRAM makes recursive reasoning probabilistic, letting a 10M-parameter model beat deterministic baselines on Sudoku, ARC-AGI, and multi-solution puzzles by sampling multiple latent reasoning paths in parallel.
- GRAM makes recursive latent reasoning probabilistic by injecting learned stochastic noise at each transition step, enabling multiple solution paths from the same input.
- With only 10M parameters, GRAM scores 97.0% on Sudoku-Extreme, 52.0% on ARC-AGI-1, and 11.1% on ARC-AGI-2, beating all deterministic recursive baselines.
- Width-based inference scaling (parallel trajectory sampling) lets GRAM with 20 samples at 16 steps beat TRM at 320 steps, a new scaling axis for recursive models.
- Deterministic recursive models collapse to ≤36.1% solution coverage on multi-solution tasks; GRAM reaches 90%+ coverage by exploring diverse latent trajectories.
- The same model generates valid Sudoku boards unconditionally with 99.05% validity using 16 steps, outperforming D3PM baselines that need 1000 steps and 5x more parameters.
- Code is coming soon; the paper and project page are available now.
Most of the recent excitement around reasoning models has focused on one axis: making models think longer by generating more tokens. Chain-of-thought, tree-of-thought, and latent reasoning approaches all extend computation by appending to a sequence. But a quieter research thread has been exploring a fundamentally different mechanism: recursive latent refinement, where a compact model repeatedly updates a hidden state rather than growing an output. A new paper from KAIST, NYU, and Mila introduces GRAM (Generative Recursive reAsoning Models), which takes this idea further by making the recursive process probabilistic.
The single-trajectory problem
Recursive Reasoning Models (RRMs) like HRM, TRM, and Looped Transformers work by repeatedly applying a shared transition function to a latent state. Each pass refines the state, and the final state is decoded into an answer. The appeal is efficiency: a small model can do deep computation by reusing the same weights many times, decoupling reasoning depth from parameter count.
The catch is that these models are entirely deterministic. Given the same input and the same initialization, they always follow the same latent trajectory and converge to the same prediction. This is fine when a problem has one right answer, but it creates a structural failure mode: the model has no way to explore alternative solution strategies, maintain uncertainty, or recover from a bad early trajectory. On problems with multiple valid solutions, deterministic recursive models exhibit mode collapse, converging to one answer and ignoring the rest of the solution space.
Stochastic guidance at every step
GRAM's core idea is to inject learned stochasticity into the recursive transition itself. Instead of computing a deterministic next state, GRAM computes a deterministic proposal u_t, then samples a Gaussian perturbation ε_t conditioned on that proposal, and adds them together:
This story is for Pro members
You've reached the end of the free preview. Upgrade to AlphaSignal Pro to read the full article - and everything else behind the paywall.
Comments
