EPFL's SFPRM Beats FPRM on Sudoku With 63% Fewer Reasoning Steps

Weight-tied transformers reuse one block across multiple iterations, giving small models adaptive reasoning depth without adding parameters. Simple inputs can halt after a few passes, while harder inputs receive more computation. A SFPRM preprint from EPFL’s LIONS lab addresses two persistent problems in this design: deciding when to stop and preventing the recurrent hidden state from growing without bound. Spherical Fixed-Point Reasoners, or SFPRM, constrain the loop to a unit hypersphere, turning each update and halting decision into bounded angular geometry.

Why Euclidean Halting Gets Messy

FPRM, SFPRM’s direct predecessor, treats a weight-tied transformer block as a damped fixed-point solver. A fixed point is a hidden state that remains unchanged by another iteration. FPRM stops when the norm of the update falls below a threshold, avoiding a separately trained halting network. The approach reached 94.2% accuracy on Sudoku-Extreme with about 7 million parameters.

FPRM’s residual stream occupies unbounded Euclidean space, so its norm can grow as the loop runs. The boundedness proof must coordinate four residual-rescaling coefficients, and the halting residual needs an additional normalization factor so thresholds remain comparable across iterations. Those mechanisms add coupled hyperparameters to training and make the stopping rule depend on a moving scale.

A Bounded Loop by Construction

SFPRM replaces the backbone with nGPT, a normalized transformer whose hidden states and parameter vectors lie on unit spheres. Each sublayer applies a learned directional update and then renormalizes the result. Hidden-state norm explosion therefore cannot occur, although bounded states can still oscillate or fail to converge.