Sesame's CSM-1B Brings the Viral Maya Voice Demo to Open-Source

Sesame's CSM-1B has quietly become one of the most downloaded speech models on Hugging Face, and it is more than another text-to-speech drop-in. It is the open base model behind the Maya voice demo that went viral for sounding disarmingly human, and it now ships natively in the Transformers library with tooling for batched inference, CUDA graph compilation, and fine-tuning.

Speech that remembers what was just said

CSM (Conversational Speech Model) generates RVQ audio codes from text and audio inputs. The architecture pairs a Llama backbone with a smaller audio decoder that produces Mimi audio codes. Traditional TTS systems generate speech from standalone text, whereas CSM-1B is optimized for conversational contexts and leverages previous dialogue turns to produce more natural and coherent output.

Under the hood, two LLaMA-style autoregressive transformer decoders do the work: a backbone that predicts the first codebook token, and a depth decoder that generates the remaining tokens. Both rely on Mimi, the pretrained codec from Kyutai, to encode speech into discrete codebook tokens and decode them back into audio. RVQ stands for residual vector quantization, a compression scheme that stacks layers of discrete tokens where each layer refines the previous one. Predicting those tokens autoregressively is what lets the model behave like a language model for sound.