---
title: "NVIDIA's Nemotron 3.5 ASR Streams 40 Languages at 17x More Concurrent Users"
subtitle: "NVIDIA released a 600M cache-aware streaming ASR model covering 40 language-locales, and it now ships as GGUF quants that run on a laptop."
url: "https://alphasignal.ai/news/nvidia-s-nemotron-3-5-asr-streams-40-languages-at-17x-more-concurrent-users"
author: "AlphaSignal Newsroom"
author_url: "https://alphasignal.ai/editorial-team"
company: "AlphaSignal"
company_url: "https://alphasignal.ai/company/alphasignal"
published: "2026-06-07T00:46:51.000Z"
updated: "2026-08-27T21:50:29.201Z"
type: "Model"
topics:
  - "Audio"
  - "Gpus"
subtopics:
  - "Speech To Text"
  - "Quantization"
  - "Edge Deployment"
upvotes: 3507
source_url: "https://huggingface.co/handy-computer/nemotron-3.5-asr-streaming-0.6b-gguf"
comments: 0
truncated: true
---

# NVIDIA's Nemotron 3.5 ASR Streams 40 Languages at 17x More Concurrent Users

> NVIDIA released a 600M cache-aware streaming ASR model covering 40 language-locales, and it now ships as GGUF quants that run on a laptop.

A community port of NVIDIA's new multilingual streaming speech recognition model just crossed [1.9 million downloads on Hugging Face](https://huggingface.co/handy-computer/nemotron-3.5-asr-streaming-0.6b-gguf), packaged as GGUF quantizations that run through a llama.cpp-style transcribe binary. The underlying model, [nvidia/nemotron-3.5-asr-streaming-0.6b](https://huggingface.co/nvidia/nemotron-3.5-asr-streaming-0.6b), is a 600M parameter cache-aware FastConformer-RNNT that transcribes 40 language-locales from a single checkpoint while streaming audio in chunks as small as 80ms.

What makes this release notable is how much it folds into one small package: low-latency streaming, wide language coverage, punctuation, capitalization, and automatic language detection in a model the size of a Whisper-tiny download. NVIDIA reports the architecture serves 17x more concurrent streams than the previous multilingual baseline.

## What actually shipped

Nemotron 3.5 ASR extends nemotron-speech-streaming-en-0.6b by adding language-ID prompt conditioning, supporting transcription across 40 language-locales from a single 600M parameter checkpoint with native punctuation and capitalization, plus runtime-configurable chunk sizes of 80ms, 160ms, 320ms, 560ms, and 1120ms. The GGUF release from handy-computer wraps that checkpoint for [transcribe.cpp](https://github.com/handy-computer/transcribe.cpp), a C++ inference runtime in the spirit of whisper.cpp.

The quantization ladder is where this becomes practical for local use:

* **F32** at 2.38 GB, **7.97%** WER on FLEURS English test
* **F16** at 1.19 GB, **7.97%** WER
* **Q8\_0** at 716 MB, **7.88%** WER
* **Q6\_K** at 593 MB, **8.02%** WER
* **Q5\_K\_M** at 534 MB, **8.15%** WER
* **Q4\_K\_M** at 473 MB, **8.49%** WER

The reference NeMo baseline on the same manifest is 7.99%, so Q8\_0 matches or slightly beats the full-precision reference within noise. On LibriSpeech test-clean the F32 preset lands at 3.04% against a NeMo reference of 3.03%. Quantization costs almost nothing here until you drop to 4 bits.

## Why cache-aware streaming changes the math

Traditional streaming ASR uses buffered inference: every few hundred milliseconds you re-feed a sliding window of audio through the encoder, which means recomputing frames you already processed. Cache-aware streaming, described in the

_Preview only. Full text requires an AlphaSignal Pro account._

## Sources

- [handy-computer/nemotron-3.5-asr-streaming-0.6b-gguf](https://huggingface.co/handy-computer/nemotron-3.5-asr-streaming-0.6b-gguf)
- [nvidia/nemotron-3.5-asr-streaming-0.6b](https://huggingface.co/nvidia/nemotron-3.5-asr-streaming-0.6b)
- [NVIDIA Nemotron 3.5 ASR API on Together AI](https://www.together.ai/models/nvidia-nemotron-35-asr)
- [Multilingual speech-to-text on your laptop: NVIDIA's Nemotron 3.5 ASR](https://livekit.com/blog/nemotron-3.5-asr-multilingual-teleprompter)
- [Stateful Conformer with Cache-based Inference](https://arxiv.org/abs/2312.17279)