NVIDIA's Nemotron 3 Ultra Runs 8x Faster Than Rival Open Models
NVIDIA's Nemotron 3 Ultra is a fully open 550B-parameter MoE model delivering 5x faster inference and 30% lower agentic task costs than comparable open models
Takeaways
- New open model: NVIDIA released Nemotron 3 Ultra, a 550B-parameter MoE model (55B active) built for long-running agentic workflows.
- Speed lead: Delivers 400+ tokens/sec and 5x higher throughput than comparable open frontier models like Kimi K2.6 and GLM-5.1.
- Cost efficiency: Completes agentic tasks (SWE-bench, Terminal Bench 2.0) using 30% fewer tokens than comparable open models.
- Top US open model: Scores 48 on the Artificial Analysis Intelligence Index -- highest of any US open-weight model, though behind China's Kimi K2.6 (54).
- Novel training: Uses Multi-Teacher On-Policy Distillation (MOPD) with 10+ specialized teacher models including DeepSeek, Qwen3, and GPT-OSS-120B.
- Fully open: Weights, datasets, RL environments, and training recipes all released under the permissive OpenMDW-1.1 license on Hugging Face.
NVIDIA just dropped its biggest open model yet, and the pitch is unusually specific: not a chatbot, not a benchmark chaser, but a reasoning engine built to run inside long-horizon agent pipelines without burning your GPU budget. Nemotron 3 Ultra is a 550 billion parameter Mixture-of-Experts model with 55 billion active parameters per token, announced at Jensen Huang's Computex keynote and released as fully open weights shortly after.
The problem it's solving
Modern agent workflows are expensive in a very specific way. Single-turn chatbots are evolving into long-running agents that reason, maintain context, use tools, and run efficiently across many turns. Every planning step, tool call, sub-agent invocation, and observation gets fed back into the model, and token counts compound fast. Within any agent workflow, most calls are routine, but a critical subset demands deeper reasoning -- sustaining architectural decisions across coding sessions, synthesizing contradictory evidence across hundreds of research sources, or verifying chip designs across thousands of constraints.
NVIDIA did not build this to win single-turn chatbot comparisons. It built it to orchestrate agents that plan, call tools, delegate to sub-agents, read observations, and recover from errors across hundreds of turns, while burning fewer tokens than comparable open models.
Speed is the headline number
Nemotron 3 Ultra scores 47.7 on the Artificial Analysis Intelligence Index, well ahead of the next strongest US open weights models -- Gemma 4 31B (39.2), Nemotron 3 Super (36.0), and gpt-oss-120b (33.3) -- but behind the Chinese-led open weights frontier (Kimi K2.6 at 53.9). The intelligence gap versus Kimi is real, but the throughput story is where Ultra pulls away from the pack.
Nemotron 3 Ultra was measured at over 400 output tokens per second from a pre-release deployment on BlackBox AI. Peer models in its size class from China-based labs such as DeepSeek and Moonshot (Kimi) are generally served at speeds of 50-100 tokens per second in the market today. That 4-8x speed advantage is not a benchmark artifact -- it changes what's practically possible in an interactive agent loop.
In experiments on SWE-bench and Terminal Bench 2.0, Ultra completed benchmarks using fewer total tokens and fewer tokens per turn than comparable models, lowering the cost for agentic tasks by up to 30%.
Five architectural bets that make this work
The speed and efficiency aren't accidental. NVIDIA made a set of deliberate architectural choices that compound on each other:
- Hybrid Mamba-Transformer layers: Mamba (a state-space model architecture that processes sequences more efficiently than standard attention) handles long-context workloads, while Transformer layers preserve precise recall when agents need to retrieve specific facts from large context windows.
- LatentMoE: LatentMoE improves accuracy through more efficient expert routing, letting the model handle workflows spanning reasoning, code generation, tool calls, and domain-specific logic without activating the full 550B parameters every time.
- Multi-Token Prediction (MTP) layers: MTP layers enable faster inference through native speculative decoding -- the model predicts multiple future tokens in a single forward pass, which is especially valuable for long outputs and multi-turn workflows.
- NVFP4 quantization: The model is pretrained in NVFP4, a 4-bit floating point format that runs on Hopper, Blackwell, and Ampere GPUs from a single checkpoint, delivering up to 5x higher throughput over BF16 on Blackwell.
- 1M token context window: The model supports a 1 million token context window, and outperforms other open models on the RULER long-context benchmark at that length.
A new training trick: MOPD
The most novel part of the training pipeline is something NVIDIA calls Multi-Teacher On-Policy Distillation (MOPD). Standard distillation trains a smaller student model to mimic a single larger teacher. MOPD scales this up dramatically.
Multi-Teacher On-Policy Distillation enables continuous improvement and domain specialization by training Nemotron 3 Ultra with dense feedback from over ten domain-specific teacher models. Each teacher is itself a specialist -- trained on a specific domain like code, math, or legal reasoning -- and scores the student's outputs in its area of expertise. The student generates its own attempts during training (on-policy), which prevents the model from just memorizing teacher outputs.
The teacher pool includes Qwen3 variants, DeepSeek V3, R1, and V4-Pro, GPT-OSS-120B, GLM-5, Mistral variants, and phi-4. NVIDIA distilled from this broad set of strong open models, which is only legally clean because Ultra is itself an open model with a permissive license and those teachers permit it.
The full training stack: pre-training runs on roughly 20 trillion tokens with a September 2025 cutoff using Megatron-LM; supervised fine-tuning with a May 2026 cutoff focuses on code, math, science, tool calling, and instruction following; reinforcement learning uses asynchronous GRPO across math, code, science, and multi-step tool use environments.
Where it's strong, where it's not
The benchmark picture is nuanced. Ultra leads on instruction following (82% on IFBench vs. 77% for GLM-5.1 and 74% for Kimi K2.6), agent productivity (tied with Kimi at 91% on PinchBench), and long context (95% on RULER at 1M tokens, while GLM and Kimi cap out at 256K). But it trails on long-horizon planning (33% on EnterpriseOps-Gym vs. 40% for GLM-5.1) and coding benchmarks (54% on Terminal-Bench 2.0 vs. 67% for Kimi K2.6).
The honest summary: Kimi K2.6 scores 54 on the Intelligence Index compared to Ultra's 48 and leads on several reasoning benchmarks including GPQA Diamond. However, Ultra delivers 300+ tokens per second throughput on NVIDIA hardware, offers 30% lower per-task cost due to token efficiency, and is the clear choice for US-based enterprises with data residency requirements.
What's actually open
NVIDIA is releasing substantially more than just weights. The full open release includes:
- The NVFP4 quantized post-trained model and the BF16 post-trained checkpoint on Hugging Face
- Pre-training datasets including 173B tokens of fresh code data from GitHub through September 30, 2025
- 10M new SFT samples, 1M new RL tasks across multiple domains, and 15 new RL environments
- Full training recipes for SFT LoRA, full SFT, GRPO, and MOPD via NeMo-RL
The licensing has also been updated. Nemotron models are moving to OpenMDW-1.1, the Linux Foundation's permissive license for open AI model distributions, covering architecture, parameters, documentation, and software under a single framework.
How to run it today
Self-hosting requires serious hardware. The BF16 weights of a 550B-parameter model require multi-GPU configurations (8xH100 or equivalent). For most teams, the managed inference path is more practical.
Pricing on DeepInfra is $0.37/M input and $1.08/M output -- better than median for this size tier. The model is also available on OpenRouter, Perplexity Pro, Anaconda, and build.nvidia.com as an NVIDIA NIM microservice. Cloud providers including Amazon SageMaker JumpStart, Google Cloud, Microsoft Foundry, and Oracle Cloud all have it available at launch.
For agent harness integration, NVIDIA has published getting-started recipes for Cline, CrewAI, LangChain, OpenHands, OpenCode, and a dozen others. Fine-tuning is supported via LoRA, full SFT, and RL using the NeMo libraries.
The bigger shift this signals
This launch positions NVIDIA as a comprehensive AI platform provider, beyond its traditional role as a chipmaker. The Nemotron 3 Ultra release is not just a model -- it comes bundled with NVIDIA OpenShell (a secure runtime for autonomous agent code execution), NemoClaw (an open-source blueprint tying the agent stack together), a new multilingual ASR model, and a 4B content safety guardrail model.
The conversation has moved past whether agents are possible. The harder problem is making them reliable, controllable, efficient, and usable inside existing organizations. Nemotron 3 Ultra is NVIDIA's answer to that problem -- and the fact that it's fully open, with transparent training data and reproducible recipes, is a direct challenge to the assumption that frontier-grade agentic models have to come from a closed API.