NVIDIA's Nemotron 3.5 Transcribe Matches Top Speech AI With 6x Less Compute
NVIDIA's new 30B mixture-of-experts transcription model matches top open-weight accuracy at 2.8% WER while using 6x fewer active parameters than rivals.
- NVIDIA released Nemotron 3.5 Transcribe, an open-weight 30B MoE speech model with 4B active parameters.
- Ties Voxtral Small for #1 open-weight on Artificial Analysis at 2.8% WER using 6x fewer active params.
- Single checkpoint handles transcription, diarization (8 speakers), translation, and audio Q&A across 40 languages.
- Hosted on DeepInfra at $2.00 per 1,000 minutes with 75x real-time speed factor.
- Non-streaming only, built for recorded audio up to one hour per clip.
- Shipped with Granary v2 open dataset and fine-tuning recipes under OpenMDW-1.1 license.
NVIDIA has released Nemotron 3.5 Transcribe, a 30-billion-parameter open-weight model for recorded speech. On the Artificial Analysis benchmark, it records a 2.81% word error rate, close to Mistral’s Voxtral Small at 2.77%. Both round to 2.8%, while Nemotron activates about 4 billion parameters per token compared with Voxtral’s 24 billion.
Sparse routing cuts active compute
Nemotron combines a speech encoder with a hybrid Mamba-Transformer mixture-of-experts language model. A router sends each token through a subset of the model’s experts, so roughly 4 billion of its 30 billion parameters participate in each step. This design brings large-model instruction following to speech recognition while limiting per-token computation.
Active parameter count remains a proxy for computational cost. Actual latency, throughput, power use, and serving cost depend on hardware, numerical precision, batching, sequence length, and runtime optimization.
Four speech jobs share one checkpoint
Nemotron consolidates several capabilities that often require separate speech, diarization, translation, and language models:
- Transcription: Speech recognition across 40 languages.
- Translation: Translation between English and 31 supported languages.
- Speaker attribution: Labels for as many as eight speakers across 13 languages.
- Audio understanding: Question answering and summarization across collections containing up to 100 hours of audio.
Natural-language prompts can specify the task, output format, expected terminology, proper names, and other context. Developers can use those instructions to improve recognition of specialized vocabulary before committing to fine-tuning. Persistent adaptation to a domain, accent, or organization-specific corpus still benefits from training on representative data.
A rounded tie with a wide sparsity gap
Word error rate, or WER, measures substitutions, deletions, and insertions relative to a reference transcript. Lower percentages indicate fewer transcription errors, although results from different benchmark suites are not directly comparable.
| Benchmark | Nemotron result | Comparison |
|---|---|---|
| Artificial Analysis AA-WER | 2.81% | Voxtral Small scores 2.77%. The models tie when rounded to one decimal place, with about 4B versus 24B active parameters. |
| AA-AgentTalk | 1.8% | The next-best open-weight result is 2.3%. AgentTalk represents half of the composite AA-WER index. |
| Earlier open-weight models | 2.81% | NVIDIA Canary Qwen 2.5B scores 4.3%, while Thinking Machines’ Inkling scores 3.5%. |
| Hugging Face Open ASR | 5.33% average WER | NVIDIA reports first place for English. This suite uses different data from Artificial Analysis. |
| Speaker attribution | Two first-place results | NVIDIA reports leading results on two attribution benchmarks, separate from transcript WER. |
The 4B figure does not shrink storage
Self-hosting requires access to all 30 billion weights even though each token activates only a subset. Approximate raw weight storage is 60 GB at 16-bit precision, 30 GB at 8-bit precision, or 15 GB at 4-bit precision. Deployments also need memory for activations, caches, audio features, and runtime overhead.
NVIDIA lists deployment targets ranging from Jetson AGX Thor and DGX Spark to Blackwell data-center GPUs. Feasibility on smaller systems depends on quantization, available unified memory, and runtime support. NVIDIA also reports that a Blackwell Server Edition GPU using NVFP4 precision and 4:1 weight compression can process 100 hours of audio in under three minutes. That figure measures batched throughput.
Hosted inference is already available
The downloadable weights use the OpenMDW-1.1 license, which teams should review for use and redistribution terms. DeepInfra also offers hosted inference, listing a 75-times real-time speed factor and a price of $2 per 1,000 audio minutes.
At 75-times real time, a one-minute recording takes about 0.8 seconds and a one-hour recording takes about 48 seconds under the provider’s test conditions. Batch size, file length, concurrency, region, and cold starts can change production performance, so endpoint benchmarks should include representative traffic.
Recorded audio sets the boundary
Transcription supports recordings up to one hour and does not provide live streaming. NVIDIA’s separate 600-million-parameter streaming Nemotron 3.5 ASR is better suited to live captions and voice-agent input. A combined system can use the streaming model during a conversation and run Nemotron 3.5 Transcribe afterward for higher-accuracy transcripts, speaker labels, summaries, and analysis.
The stated 100-hour capacity applies to audio question answering across large collections. It does not raise the one-hour limit for an individual transcription request.
Prompting and fine-tuning move different levers
NVIDIA paired the model with Granary v2, an updated open-source speech dataset and fine-tuning pipeline. On a difficult internal benchmark, the base model scored 30.8% WER, while careful prompting reduced the result to 20.8%.
A version fine-tuned on the Eka Care Indian medical speech benchmark reached 11.8% WER. That result comes from a different corpus, so it cannot be compared directly with the internal benchmark. Together, the tests show two available adaptation paths: prompts can supply immediate context, while fine-tuning can target recurring vocabulary, accents, and acoustic conditions.
Where the model fits
| Workload | Fit | Deployment note |
|---|---|---|
| Meeting and podcast transcription | Direct | Combines transcripts, speaker labels, summaries, and audio questions in one checkpoint. |
| Post-call analytics | Direct | The 1.8% AgentTalk result supports evaluation for recorded agent conversations. |
| Medical or specialized dictation | Requires validation | Start with terminology prompts, then fine-tune if errors remain systematic. |
| Multilingual archives | Direct for supported languages | Confirm language-specific accuracy and translation direction on representative recordings. |
| Live captions and voice agents | Requires a streaming front end | Use a streaming ASR model during the session and Nemotron for post-processing. |
| Memory-constrained deployment | Configuration-dependent | Plan capacity around all 30B weights, quantization overhead, and runtime memory. |
For recorded-audio systems, Nemotron’s main advantage is the combination of competitive transcription accuracy, sparse active computation, multilingual support, speaker attribution, and audio understanding in a self-hostable checkpoint. Production evaluations should measure full memory use, end-to-end latency, language-specific accuracy, and domain error rates rather than relying on active parameter count alone.
Comments
