Meta's Muse Voice Transcribe Beats Every Rival With 3.1% Error Rate

Meta Superintelligence Labs has released Muse Voice Transcribe, a real-time audio perception model that folds streaming speech recognition, speaker diarization, and endpointing into a single autoregressive multimodal LLM. On Artificial Analysis's AA-WER streaming benchmark it hits a 3.1% word error rate, ahead of Cartesia Ink-2 (3.4%), ElevenLabs' Scribe v2 Realtime (3.6%), GPT Live Transcribe (3.9%), and Gemini 3.5 Transcribe Live (4%). It also takes the top spot on public diarization benchmarks while running fast enough to power live dictation on a Mac.

The model belongs to the Muse Spark family and is available today. You can hit it through the Meta Model API for $3 per 1,000 audio-minutes (about $0.18 per hour), and it already powers dictation in Meta AI for Mac and Muse Code, where holding the Fn key routes audio into any application on screen.

One model, three jobs

Real-time transcription systems have historically been assembled from separate components: a streaming ASR model, a diarization pipeline, and a voice activity detector for endpointing. Muse Voice Transcribe collapses that stack. Meta says the model transcribes speech as it happens, separates 20-plus voices across recordings, and determines when someone has finished talking, all without a separate post-processing step.

Under the hood, it is an autoregressive multimodal LLM with a streaming twist. Audio is chunked at 12.5 Hz (80 ms per chunk), and each chunk is compressed into one soft token. At every step the model picks between two actions: emit a text token, or emit a special <|next_audio|> token that gets replaced with the next 80 ms of audio on the next step. When the input stops, an <|empty_audio|> token tells the model to drain whatever text is left in its buffer.

Diarization and endpointing are bolted on with additional special tokens rather than separate networks:

• <|start_of_turn|> marks a potential speaker switch and is emitted immediately.
• <|speaker_A|> through <|speaker_Z|> tags identify who spoke, and are deferred to the end of the turn where more context is available.
• <|speech_onset|> and <|speech_endpoint|> bracket each utterance so a downstream assistant knows exactly when to start reasoning.

A diarized transcript token stream literally looks like this:

<|start_of_turn|>Hello, how are you doing?<|speaker_A|>
<|start_of_turn|> Did anything fun over the weekend?<|speaker_A|>
<|start_of_turn|> Hey I'm good!<|speaker_B|>

Training multiplexes ASR, diarization, and endpointing objectives together, with separate rewards layered on top of the ASR loss for each auxiliary task.

Learning when to commit

The most interesting piece is how the model manages latency. Streaming ASR has a fundamental tension baked in: waiting longer before committing a word gives you more acoustic and linguistic context and lowers error rate, but it also raises the perceived delay. Most systems pick a fixed lookahead and live with the tradeoff.

Muse Voice Transcribe learns an adaptive delay instead. Because the model itself decides whether to keep listening or emit text at every 80 ms chunk, delay becomes a policy that can be trained. Meta uses reinforcement learning where the word error rate reward and a delay reward are combined multiplicatively, so the model waits longer on ambiguous words and commits sooner on easy ones. According to the research post, this produces a new Pareto frontier on the speed versus accuracy curve for time-to-final-transcription.

Multilingual and multi-speaker by default

The model was trained across more than 70 languages, with 25 validated at launch, and it supports audio longer than an hour plus native code-switching within or between sentences. Meta's demos include a stretch of Mandarin-English code-switching packed with technical terms (Ollama, GGUF, quantization, speculative decoding) where context biasing stays active to lock in domain vocabulary. Language, keyword, and context biasing can further sharpen recognition.

On diarization, Meta reports a 17.5% average error rate across AMI-IHM, AMI-SDM, and VoxConverse, compared to five other systems ranging from 21.1% to 28.6%. The headline demo puts eight speakers in the same room, some switching languages mid-sentence, with tags applied live.

Use cases and catches

The obvious applications are the ones existing ASR APIs already serve, but with cleaner output and lower latency:

• Meeting transcription and note-taking where speaker labels matter.
• Voice-driven agents that need reliable endpointing so they don't cut users off or wait too long to reply.
• Live captioning for multilingual conversations.
• Dictation into arbitrary desktop apps, which is what the Mac Fn-key integration demonstrates.
• Long-form audio (interviews, podcasts, courtroom-style recordings) with 20+ participants and no post-processing pass.

There are real limitations to flag. The Artificial Analysis benchmark only covers English speech, so the reported 3.1% WER says little about how the other 24 validated languages perform in practice. Unlike Meta's Muse Glimmer models, open weights don't appear to be on the roadmap, so anyone who wants to run it on-device or fine-tune it is out of luck.

Where this leaves the market

Real-time transcription is a crowded field. OpenAI's Whisper set a high bar for open-source quality, Google's Chirp models power speech recognition across its cloud platform, and companies like AssemblyAI and Deepgram have built entire businesses around real-time APIs with speaker identification. Meta's pitch is a single model that handles ASR, diarization, and endpointing jointly, learns its own latency policy, and undercuts the field on price.

The deeper story is architectural. Treating audio perception as "an LLM that emits either audio-continue tokens or text tokens" is a clean way to fold speech into the same paradigm as text generation. Diarization and endpointing become more vocabulary. Adaptive delay becomes another RL objective. If this design generalizes, expect follow-up models in the Muse Spark family that add more perception modalities using the same tokenized-decision recipe, especially as Meta pushes toward the always-listening AI glasses scenario the launch demo hints at.