NVIDIA's Nemotron 3 Nano Omni Handles Video, Audio, and Docs on One GPU
NVIDIA's new 30B mixture-of-experts model handles text, image, audio, and video jointly, running on a single L40S in FP8 with 256k context.
Takeaways
- NVIDIA released Nemotron 3 Nano Omni, a 30B MoE model handling video, audio, image and text.
- Hybrid Mamba-Transformer with 23 Mamba layers, 128 experts top-6 routing, 3B active parameters.
- FP8 variant runs on a single 48GB L40S with 256k context and stays within 0.4 points of BF16 accuracy.
- Beats Qwen3-Omni on MMLongBench-Doc (57.5), Video-MME (72.2), VoiceBench (89.4) and OSWorld (47.4).
- Reports 7.4x throughput on multi-doc and 9.2x on video versus other open omni models.
- Free commercial use under the NVIDIA Open Model Agreement, deployable on vLLM, SGLang, TensorRT-LLM.
NVIDIA has released Nemotron 3 Nano Omni, an open-weights multimodal model that unifies video, audio, image, and text understanding in a single 30B mixture-of-experts backbone with roughly 3B active parameters per token. The model already has close to a million downloads on Hugging Face, and it ships in three precisions, with the FP8 checkpoint fitting on a single 48GB L40S.
NVIDIA targets five workload classes: real-world document analysis, automatic speech recognition, long audio-video understanding, agentic computer use, and general multimodal reasoning. That framing matters because the model is aimed squarely at the kind of messy enterprise inputs, hundred-page PDFs, meeting recordings, GUI screenshots, that most open models handle poorly.
What is actually in the box
The architecture is a hybrid Mamba-Transformer mixture of experts. The backbone interleaves 23 Mamba selective state-space layers for efficient long-context processing, 23 MoE layers with 128 experts and top-6 routing plus a shared expert, and 6 grouped-query attention layers to preserve global interaction. Vision comes from the C-RADIOv4-H encoder, and audio from Parakeet-TDT-0.6B-v2, both connected to the language model via lightweight 2-layer MLP projectors.
Key specs:
- 31B total parameters with about 3B active per token, 256k max context, and text-only output
- Inputs: mp4 video up to 2 minutes, wav or mp3 audio up to 1 hour, jpeg/png images, and text
- Precisions: BF16 at 62 GB, FP8 at 33 GB, and NVFP4 at 21 GB
- Minimum GPU: one H100 80GB for BF16, one L40S 48GB for FP8, and one RTX 5090 32GB for NVFP4
- Supports JSON output, chain-of-thought reasoning, tool calling, and word-level timestamps for transcription
The license is the NVIDIA Open Model Agreement, weights are free, and commercial use is permitted.
Where it actually wins
The benchmark story is striking, particularly against the previous Nemotron Nano VL V2 and against Qwen3-Omni 30B-A3B, the closest open competitor. On OCRBenchV2 English it scores 67.04 versus 54.8 for Nano VL V2, on OSWorld computer-use it hits 47.4 versus 11.1, and Charxiv chart reasoning jumps from 41.3 to 63.6. On MMLongBench-Doc it reaches 57.5 versus 49.5 for Qwen3-Omni, and on Video-MME it scores 72.2 versus 70.5.