Boson AI's Higgs TTS 3 Brings Human Emotion to Open Voice Agents

Boson AI's 4B open-weight TTS model handles 100+ languages with zero-shot voice cloning, inline emotion control, and sub-second streaming for real voice agents.

·
·
·
Boson AI's Higgs TTS 3 Brings Human Emotion to Open Voice AgentsPRO
  • Boson AI released Higgs TTS 3, a 4B TTS model built on Qwen3-4B for conversational voice agents.
  • Supports 100+ languages with single-digit WER/CER, plus zero-shot cross-lingual voice cloning from a short reference clip.
  • Inline control tokens for 21 emotions, style, prosody, pauses, and sound effects, embedded directly in input text.
  • Streams audio via Server-Sent Events, 617 ms first-audio latency and 14.74 req/s on a single H100 through SGLang-Omni.
  • Beats Qwen3-TTS-1.7B, Fish Audio S2 Pro, IndexTTS-2 on Emergent TTS and multilingual benchmarks.
  • Research and non-commercial license, but a Creator Use Grant lets creators monetize output free with attribution.

Voice agents have a boring-voice problem. Most open text-to-speech models sound like a polite narrator reading a Wikipedia page, not a person mid-conversation. Higgs TTS 3 from Boson AI is a 4B-parameter model built to close that gap, released on Hugging Face with weights, serving recipes, and a license that lets solo creators monetize output for free.

How the architecture works

Higgs TTS 3 runs on a Qwen3-4B autoregressive decoder backbone. Audio is encoded by the Higgs Tokenizer into 8 codebooks at 25 fps, staggered via a delay pattern, then mapped to backbone hidden states through a multi-codebook fused embedding. Output codes pass through a multi-codebook fused head, are de-delayed, and decoded back to 24 kHz waveform.

Rather than waiting for a complete sentence, the model begins synthesis as text tokens arrive, continuing as the stream grows while preserving stable delivery. That matters when you pipe tokens directly from an LLM into a voice pipeline and want the user to hear audio before the model finishes generating.

Inline tags that reshape delivery

Control is exposed through inline tokens dropped directly into the input string, with four categories:

  • Emotion tags like <|emotion:amusement|>, <|emotion:sadness|>, and <|emotion:awe|>, covering 21 states
  • Style tags for singing, shouting, and whispering
  • Sound effects such as <|sfx:laughter|>, <|sfx:sigh|>, and <|sfx:sneeze|>, each paired with a written onomatopoeia like "Haha" or "Achoo" that gives the model an acoustic cue

Pro article

This story is for Pro members

You've reached the end of the free preview. Upgrade to AlphaSignal Pro to read the full article - and everything else behind the paywall.

Trending
  • No trending articles

Comments

avatar

Next Reads