Kyutai’s Voice of Reason thinks while it speaks

Kyutai has released Voice of Reason, a speech-to-speech model that can hear a math word problem and answer aloud without routing the task through a separate text LLM. Built on GLM-4-Voice-9B, its strongest checkpoint raises reported GSM8K accuracy from 27.3% to 77.1%.

The model generates private text reasoning alongside audio tokens, using the time occupied by spoken output to prepare later parts of its answer. That design reduces the long initial pause common in pipelines that transcribe speech, run a complete text reasoning pass, and synthesize the result.

Reasoning in the playback window

Speech systems often combine automatic speech recognition, a text LLM, and text-to-speech synthesis. This cascade can reason well, but each stage adds latency, and the text model may finish a long reasoning trace before speech synthesis begins.

The STITCH paper from Chiang and co-authors exploits a timing difference between generation and playback. Spoken audio takes longer to play than its underlying tokens take to generate, leaving a computation window in which the model can produce hidden reasoning tokens for the next speech segment.

The paper reports latency comparable to speech baselines that omit hidden reasoning, alongside gains of about 15% on math reasoning datasets. Kyutai applied that training and inference method to GLM-4-Voice and published the resulting weights.

Two checkpoints, two reasoning modes

The direct checkpoint’s 43-point gain over the base model indicates that the training pipeline contributes substantially to accuracy. STITCH adds a further 6.8 points by giving the model private reasoning capacity during generation.

Training the interleave

Training begins with supervised fine-tuning on stitched dialogues. Each example alternates between written reasoning segments that remain unspoken and response segments represented as text and audio tokens.

A reinforcement-learning stage then rewards correct answers to math word problems using a binary LLM judge. The task provides a narrowly defined outcome, answer correctness, instead of a broad preference score, although the reward still depends on the judge classifying responses accurately.

The published inference prompt instructs the model to generate reasoning in chunks of up to 100 tokens enclosed by [SOPR] and [EOPR] markers. Spoken sections follow an interleaving schedule of 13 text tokens and 26 audio tokens. Content inside the reasoning markers remains private, while the remaining response content is rendered as speech.

Kyutai reports the 77.1% result across 1,310 GSM8K test items, using gpt-4o-2024-11-20 to judge the written response channel. GSM8K measures answers to grade-school math word problems; the reported score covers answer correctness rather than speech quality, conversational range, or real-world latency.

Local deployment needs three components

The model card lists roughly 10 billion parameters despite the 9B checkpoint name and says the model fits on one H100 GPU in bf16. A working audio pipeline combines the following pieces:

• The Voice of Reason checkpoint, distributed under the GLM-4-Voice license.
• The GLM-4-Voice repository and its Whisper-based speech tokenizer, which converts input audio into discrete tokens.
• The THUDM/glm-4-voice-decoder checkpoint, which converts generated audio codes into a waveform.

A minimal model loader uses the custom Transformers implementation included with the repository:

from transformers import AutoModel, AutoTokenizer
import torch

REPO = "kyutai/glm-4-voice-of-reason-stitch-9b"

tokenizer = AutoTokenizer.from_pretrained(
    REPO,
    trust_remote_code=True,
)

model = AutoModel.from_pretrained(
    REPO,
    torch_dtype=torch.bfloat16,
    device_map="cuda",
    trust_remote_code=True,
).eval()

The trust_remote_code=True option executes Python supplied by the model repository. Production deployments should inspect that code and pin a reviewed revision before loading it.

Generation returns a mixed stream of text tokens and audio codes. The application removes spans enclosed by [SOPR] and [EOPR], retains the public response, and passes the audio codes to the GLM-4-Voice decoder. Kyutai also says the original project’s full-duplex demo, which supports concurrent listening and speaking, can use this checkpoint through its --model-path option.

Where the model fits

Voice of Reason targets applications that need spoken interaction and structured reasoning, including tutoring tools, phone assistants, in-car interfaces, and accessibility software. Its math-focused reinforcement learning and GSM8K evaluation provide limited evidence for open-ended conversation, other reasoning domains, multilingual performance, or noisy acoustic settings.

The release gives developers an audio-to-audio alternative to ASR, text-LLM, and TTS cascades when response latency matters. Its central technique is portable: a speech model can spend the playback interval computing later reasoning steps while continuing to deliver audio.