NVIDIA's Nemotron 3 Ultra Becomes the Most Capable Open US Model at 550B
NVIDIA's 550B open-weights model tops US rankings with 5x faster inference, built for long-running AI agents — but China's Kimi K2.6 still leads globally
NVIDIA just shipped its biggest open model yet. Nemotron 3 Ultra is a 550 billion parameter Mixture-of-Experts reasoning model, announced at Jensen Huang's Computex keynote and released days later with full weights, training data, and recipes under a permissive Linux Foundation license. The headline benchmark number is 48 on the Artificial Analysis Intelligence Index, making it the most capable open-weights model to come out of a US lab. But the more interesting story is why it was built, and what it trades to get there.
Not a chatbot. An orchestrator.
Nemotron 3 Ultra targets a specific problem: long-running agents that plan, call tools, and reason across many turns. As agents run longer, token counts grow and inference cost climbs. Ultra is designed to keep accuracy high while making that inference faster and cheaper. NVIDIA's framing is explicit: this model is not competing on single-turn chat quality. It is competing on cost per completed agentic task.
What makes Nemotron 3 Ultra different is the design goal. NVIDIA did not build this to win single-turn chatbot comparisons. According to the NVIDIA blog, it built it to orchestrate agents that plan, call tools, delegate to sub-agents, read observations, and recover from errors across hundreds of turns, while burning fewer tokens than comparable open models.
The Nemotron 3 family is tiered by design. Nano handles edge and on-device workloads. Super handles high-volume execution. Ultra sits at the top as the reasoning orchestrator, and because all three are open under the same permissive license, Ultra's outputs can legally be used to post-train the smaller models below it , something the terms of most closed frontier APIs prohibit.
The architecture is the bet
Nemotron 3 Ultra is a 55B active, 550B total parameter Mixture-of-Experts hybrid Mamba-Transformer model that leverages LatentMoE, includes MTP layers, and was pre-trained in NVFP4. Each of those pieces is doing real work.
- Hybrid Mamba-Transformer: Nemotron 3 Ultra employs a Mixture-of-Experts hybrid Mamba-Attention architecture, leading to improvements along the inference-throughput-to-accuracy frontier. While Mixture-of-Experts help Ultra achieve better accuracy per active parameter, the hybrid Mamba-Attention architecture significantly improves inference throughput by reducing attention cost and KV cache footprint.
- LatentMoE: A major innovation inside Nemotron 3 Ultra is NVIDIA's LatentMoE architecture. Traditional MoE systems route raw tokens to experts. Instead, information is compressed into latent representations, experts process compressed information, and more experts can be utilized without proportional computational cost.
- Multi-Token Prediction (MTP): MTP layers enable faster inference through native speculative decoding. The model predicts multiple future tokens in a single forward pass, cutting latency on long outputs and multi-turn workflows.
- NVFP4 precision: Training precision is NVFP4 (NVIDIA's 4-bit floating point format, native to Blackwell architecture). Approximately 85% of layers are NVFP4, while attention and projection layers are retained in BF16/MXFP8. This is designed specifically to maximize throughput on GB300 and Blackwell Ultra platforms.
The Mixture-of-Experts design activates only 55B of the 550B parameters per token , a 10x sparsity ratio. That is what lets a 550B model run at speeds that would normally require a model a fraction of its size.
A novel training method: MOPD
The most technically interesting piece of the training pipeline is something NVIDIA calls Multi-Teacher On-Policy Distillation (MOPD). MOPD enables continuous improvement and domain specialization by training Nemotron 3 Ultra with dense feedback from over ten domain-specific teacher models, supported by a massive and transparent pretraining and RL data pipeline, with fully open recipes, weights, and licensing for broad adoption and fine-tuning.
The loop works like this: as the student model generates rollouts across domains (coding, math, tool use, etc.), the matching specialist teacher scores it in its area of expertise. The whole pipeline runs asynchronously , student generation, teacher scoring, and optimization are fully pipelined. After each round, teacher models are re-initialized from the updated student, so both co-evolve. Post-training uses an enhanced pipeline involving Supervised Fine Tuning (SFT), Reinforcement Learning (RL), and Multi-teacher On-Policy Distillation (MOPD) for improved model accuracy.
Nemotron 3 Ultra was trained on 20 trillion tokens. On top of that foundation, NVIDIA added 212 billion new tokens targeting specific domain gaps: 4B tokens of synthetic legal data, 35B tokens of synthesized Wiki-based data, and 173B refreshed GitHub tokens through September 2025. The result was measurable: LegalBench proxy scores jumped from 64.6% to 74.7%, and SimpleQA from 40.2% to 50.2%.
Where it wins, and where it doesn't
According to benchmark platform Artificial Analysis, it is now the most capable open model to come out of a US lab, scoring 48 on the Artificial Analysis Intelligence Index. That puts it well ahead of every other open American model and lands it in what Artificial Analysis calls the most attractive quadrant on its chart, combining high intelligence with fast output speed.
The benchmark table from NVIDIA's own head-to-head against GLM 5.1 (744B), Kimi K2.6 (1T), and Qwen3.5 (397B) tells a nuanced story:
| Benchmark | Nemotron 3 Ultra (550B) | GLM 5.1 (744B) | Kimi K2.6 (1T) | Qwen3.5 (397B) |
|---|---|---|---|---|
| Agent Productivity (PinchBench) | 91% | 84% | 91% | 89% |
| Long-horizon Planning (EnterpriseOps-Gym) | 33% | 40% | 29% | 30% |
| Coding (Terminal-Bench 2.0) | 54% | 64% | 67% | 53% |
| Instruction Following (IFBench) | 82% | 77% | 74% | 78% |
| Long Context (Ruler @1M) | 95% | N/A (max 256K) | N/A (max 256K) | 90% |
The long-context result is the most structurally significant. With a 1M token context window (with NVFP4 quantization on Blackwell), Ultra scores 95% on Ruler @1M , while both GLM 5.1 and Kimi K2.6 top out at 256K and cannot run the test at all. For agent workloads that need to hold a large codebase or extended conversation history in context, this is a real structural advantage, not just a benchmark number.
The gaps are equally honest. On raw coding (Terminal-Bench 2.0), Ultra scores 54% against Kimi K2.6's 67%. On long-horizon planning, it trails GLM 5.1 by 7 points. Kimi K2.6 leads the overall intelligence index by a margin of 6 points (54 vs 48). Ultra wins on the axes NVIDIA optimized for , agent productivity, instruction following, long context, speed, and cost , and trails on the axes it traded away.
Speed is the real differentiator
On a pre-release DeepInfra endpoint, Artificial Analysis recorded 300+ tokens/second. Comparably-sized Chinese open models typically run at 50–100 tokens/second in available hosted deployments. That is a 3-6x throughput gap against peer models, and it comes from the combination of low active parameter count, NVFP4 precision, the Mamba hybrid, and multi-token prediction working together.
Ultra delivers 5x faster inference and lowers the cost of complex agentic tasks by up to 30% versus other open frontier models. For long-running agents that call the model thousands of times to finish one workflow, a 30% reduction in tokens per task compounds into a meaningful cost difference at scale.
Fully open, with a wide deployment footprint
Licensed under OpenMDW-1.1 (Linux Foundation permissive license), everything is publicly available: base model weights on Hugging Face, post-trained checkpoints, reward models used in RLHF post-training, NVFP4 quantized variants optimized for Blackwell, and training recipes, datasets, and evaluation code.
Nemotron 3 Ultra is available on HuggingFace, OpenRouter, ModelScope, and NVIDIA NIM. For managed API access, DeepInfra pricing sits at $0.37/M input and $1.08/M output , which is competitive for this size tier. The model also integrates with every major agent harness out of the box, including CrewAI, LangChain, OpenHands, Cline, and OpenCode.
For teams that want to fine-tune, NVIDIA ships LoRA, full SFT, and RL recipes for both H100 and GB200 configurations through the NeMo libraries. The MOPD recipe is also open, meaning teams can reproduce the multi-teacher distillation pipeline on their own domain-specific teacher models.
Here is the minimal setup to hit the model via the OpenAI-compatible NIM endpoint:
from openai import OpenAI
client = OpenAI(
base_url="https://integrate.api.nvidia.com/v1",
api_key="YOUR_API_KEY"
)
response = client.chat.completions.create(
model="nvidia/nemotron-3-ultra-550b-a55b",
messages=[{"role": "user