Skip to main content
Back

ReLAT Fixes AI's Hidden Reasoning Flaw and Cuts Token Usage by 84%

ReLAT uses a self-supervised reconstruction loop to verify latent reasoning states at test time, pushing Qwen3-8B from 56.7% to 73.3% on AIME 2024

Paper, Full Access
  • ReLAT is a self-supervised test-time training method that verifies latent reasoning states by checking if the original question can be reconstructed from them before generating an answer.
  • On Qwen3-8B, ReLAT raises AIME 2024 accuracy from 56.7% to 73.3%, a 16.6-point gain over the best open-loop latent baseline.
  • The method uses a Question → Latent Thought → Question differentiable cycle with temporary LoRA adapters that are reset after each example, requiring no labels.
  • ReLAT uses 84% fewer tokens than text-based iterative refinement (Self-Refine) while achieving 10 points higher accuracy on AIME 2025.
  • Gains are consistent across math, code, and expert QA benchmarks on Qwen3-4B/8B/14B and DeepSeek-R1-Distill-Qwen-7B.
  • The paper is available on arXiv; no public code repository has been released yet.

A new paper introduces ReLAT (Reconstruction-Guided Latent Reasoning At Test Time), a self-supervised method that addresses a fundamental blind spot in how large language models reason in latent space. The core problem it solves is deceptively simple: when a model thinks in continuous, non-text representations, there is no way to check whether those internal thoughts still faithfully capture the original question. ReLAT closes that gap by forcing the model to reconstruct the question from its own latent thoughts before generating an answer.

The Open-Loop Problem in Latent Reasoning

To understand why this matters, you need to know what latent reasoning is. Traditional chain-of-thought reasoning externalizes every intermediate step as text, which is readable and verifiable but token-expensive. A growing line of work instead keeps those intermediate steps as continuous vector representations, or KV-cache states, passing them directly between reasoning stages without ever decoding them into words. This is faster and richer, but it creates a new problem.

Latent reasoning operates in an open loop: the intermediate state is produced and consumed without any check that it still represents the query it was derived from. In text-based reasoning, a dropped constraint is visibly wrong. In latent reasoning, an opaque latent state that drops the same constraint looks no different from a faithful one, and is consumed by downstream computation as if it were reliable.

Framework comparison between Open-Loop Latent Collaboration and ReLAT's closed-loop reconstruction-guided approach

To close this loop, we need a way to test whether an opaque latent state still preserves the information needed to specify its original query. The key observation is that the query itself is the natural reference for this test. If a latent state is a faithful representation of the question, the question should be recoverable from it. If not, the latent state has lost task-relevant information.

How ReLAT Works

ReLAT operationalizes this idea as a differentiable cycle: Question → Latent Thought → Question. Before generating any answer, the model runs this loop and minimizes the reconstruction loss through the latent thought using temporary LoRA adapters (low-rank weight updates that are cheap to compute and discarded after each example). Here is the step-by-step procedure: