Skip to main content
Back

k2-fsa's OmniVoice Clones Any Voice Across 646 Languages at 40x Real Time

A 0.6B parameter zero-shot TTS model from k2-fsa covers 646 languages, clones voices from short clips, and runs 40x faster than real-time.

Pro

  • OmniVoice from k2-fsa is a zero-shot TTS model covering 646 languages in one 0.6B checkpoint.
  • Built on a diffusion language model architecture initialized from Qwen3-0.6B-Base, trained on 581,000 hours of audio.
  • Real-time factor as low as 0.025, roughly 40x faster than real time using 16-32 diffusion steps.
  • Three modes: voice cloning from 3-10s reference, attribute-based voice design, and auto voice selection.
  • Supports non-verbal tags like [laughter] plus pinyin and phoneme-level pronunciation correction.
  • Code is Apache 2.0 but pretrained weights are CC-BY-NC due to Emilia training data restrictions.

The k2-fsa team, the group behind the widely used Kaldi and sherpa speech toolkits, has released OmniVoice, a zero-shot text-to-speech system with a headline claim that is hard to ignore: 646 languages from a single 0.6B parameter checkpoint. That is roughly an order of magnitude more languages than most open TTS systems handle, and it ships with voice cloning, attribute-based voice design, and real-time factors low enough for streaming applications.

Under the hood, OmniVoice is a single-stage diffusion language model initialized from Qwen3-0.6B-Base. It generates speech in 600+ languages and clones a voice from a short reference clip without per-speaker training. Model weights, code, paper, and a Colab notebook are all live, and a Hugging Face Space lets you try it in a browser before installing anything.

The architecture, in plain terms

OmniVoice is a bidirectional Transformer trained with a masked-diffusion objective that maps text directly to multi-codebook acoustic tokens. Instead of the two-stage pipelines common in modern TTS (an autoregressive text-to-token model plus a separate acoustic decoder), it predicts audio tokens in parallel by iteratively unmasking them, similar to how discrete diffusion works for images.

That design choice has two practical consequences worth caring about:

  • Speed. Real-time factor reported as low as ~0.025 (about 40x faster than real time), using 16-32 diffusion steps. RTF measures how long synthesis takes relative to audio duration, so 0.025 means one second of speech generates in about 25 milliseconds on a suitable GPU.