Skip to main content
Back

OpenBMB's VoxCPM2 Drops Audio Tokenizers to Clone Any Voice in 30 Languages

OpenBMB released a 2B parameter tokenizer-free TTS model that covers 30 languages, designs voices from text descriptions, and streams 48kHz audio in real time.

Pro

  • OpenBMB released VoxCPM2, a 2B parameter tokenizer-free diffusion TTS model under Apache-2.0.
  • Covers 30 languages plus 9 Chinese dialects with no language tag needed at inference.
  • Outputs 48kHz studio-quality audio via AudioVAE V2 internal super-resolution.
  • Voice Design generates new speakers from plain-text descriptions of gender, age, tone, and pace.
  • Runs at RTF 0.30 on RTX 4090, 0.13 with Nano-vLLM, needs ~8GB VRAM.
  • Supports LoRA fine-tuning from 5-10 minutes of audio via the GitHub repo.

OpenBMB just dropped VoxCPM2, a 2B parameter text-to-speech system that throws out the standard playbook. Instead of converting text into discrete audio tokens the way most modern TTS pipelines do, it generates continuous speech representations directly through a diffusion autoregressive architecture. One open-source model handles 30 languages, outputs 48kHz audio, was trained on more than 2 million hours of multilingual speech, and ships under Apache-2.0 for commercial use.

Dropping the tokenizer

Most current TTS systems (VALL-E, XTTS, and similar) first quantize speech into discrete tokens with a neural codec, then train a language model to predict those tokens. That extra compression step is often where prosody flattens and voices start sounding robotic. VoxCPM2 skips that stage entirely, generating continuous acoustic frames through an end-to-end diffusion autoregressive pipeline.

The architecture stacks four components: a local encoder (LocEnc), a text-speech language model (TSLM), a residual acoustic language model (RALM), and a local diffusion transformer (LocDiT) that predicts continuous acoustic frames. The backbone reuses MiniCPM-4 weights for 2B parameters total. A companion AudioVAE V2 handles the audio side with an asymmetric trick worth flagging: it accepts 16kHz reference clips but decodes to 48kHz output, so the model performs super-resolution internally without a separate upsampler.

Describe a voice, get a voice

The headline capability is Voice Design. Write a natural-language description in parentheses at the start of your text, and VoxCPM2 synthesizes a brand-new speaker matching it. No reference audio required.