Kyutai Opens Pocket TTS Training So Anyone Can Build CPU Voice Cloning

Kyutai has opened up the entire recipe behind Pocket TTS, its 100M-parameter text-to-speech model that runs on a laptop CPU. The release covers the data pipeline, training scripts, and evaluation suite, so you can now build one of these models from scratch on your own hardware and swap in any language or voice you want.

The original checkpoint, released earlier this year, was already an interesting piece of engineering: it ships as a pip install, needs no GPU, and clones voices from about five seconds of reference audio. What's new is that Kyutai is now letting the community reproduce and modify the training process itself.

Inside the training stack

The repo contains everything needed to go from raw speech data to a working checkpoint. Kyutai reports that the loss curve hits recognizable milestones on a predictable schedule:

• Around 15k steps: babble resolves into actual words
• Around 50k steps: arbitrary text reads with word error rate under 1%
• Around 200k steps: the voice stops sounding synthetic

On eight H100s, a full run takes 10 to 20 hours. On a single strong consumer GPU, it's roughly a week. Kyutai estimates the total rental cost at under $200, or an order of magnitude less if you already own the hardware and only pay for electricity.

Why 100M parameters is enough

Pocket TTS is built on the Continuous Audio Language Model approach, which skips the usual step of quantizing audio into discrete tokens. Rather than predicting entries from a fixed codebook, the model predicts continuous audio frames directly through an audio VAE, using consistency modeling to generate each frame. Avoiding lossy compression lets CALM reach higher quality at lower computational cost than discrete audio language models, with better efficiency and fidelity than state-of-the-art discrete counterparts on both speech and music.

That design is what keeps the parameter count so low. Pocket TTS posts the lowest Word Error Rate in its class, audio quality better than the ground truth used by F5-TTS and DSM, and comparable speaker similarity, while being the only model in the comparison that runs faster than real-time on CPU. Kyutai tested on Apple M3 and Intel Core Ultra 7 165H chips.

Runtime behavior

The inference characteristics are why people are already deploying this in production side projects. From the repo:

• CPU only, no GPU build of PyTorch required
• About 200ms latency to first audio chunk, with streaming output
• Roughly 6x faster than real-time on a MacBook Air M4
• Uses only 2 CPU cores
• Handles infinitely long text inputs
• Voice cloning from a short WAV file
• Six languages: English, French, German, Portuguese, Italian, Spanish

Using it as a library is minimal:

from pocket_tts import TTSModel
import scipy.io.wavfile
tts_model = TTSModel.load_model()
voice_state = tts_model.get_state_for_audio_prompt("alba")
audio = tts_model.generate_audio(voice_state, "Hello world, this is a test.")
scipy.io.wavfile.write("output.wav", tts_model.sample_rate, audio.numpy())

Voice states can be exported to safetensors, so loading a cloned voice later is a kvcache read rather than a re-encoding pass.

What Kyutai wants people to build

The English model was trained on 88k hours of public speech data for reproducibility, but Kyutai notes that a few hundred hours is enough to get surprisingly usable results in a new language. They are explicitly inviting the community to:

1. Train Pocket TTS in new languages using the shared data pipeline
2. Add features like emotion tags or singing capability
3. Push the model faster or smaller than the reference recipe

Kyutai has promised to highlight the best community checkpoints. A Czech version is already floating around as a demo, reportedly produced by pointing a coding agent at the training scripts and letting it handle the data collection.

Where it fits in the ecosystem

Most high-quality TTS today either requires a GPU-hosted API or ships as a 1B+ parameter model. Smaller options like Kokoro are fast but weak at voice cloning. Pocket TTS sits in an unusual spot: small enough for a Raspberry Pi or a browser via WebAssembly, with cloning quality closer to the larger systems. Community ports already cover sherpa-onnx for embedded boards, MLX for Apple Silicon, ONNX Runtime Web, and a Rust/Candle build.

The practical effect is that any team wanting an in-house TTS voice can now avoid sending audio to a third-party API. Screen readers, game NPC dialogue, on-device assistants, accessibility tools for people with visual impairments or ALS, and dubbing for underrepresented languages all become tractable weekend projects rather than infrastructure decisions. The MIT license, combined with a data recipe built on public speech corpora, means the resulting checkpoints are safe to ship commercially.