Skip to main content
Back

NVIDIA's Nemotron 3 Ultra Beats Trillion-Parameter Models at 5x the Speed

NVIDIA's Nemotron 3 Ultra is a fully open 550B-parameter reasoning model that tops the US open-weights leaderboard while running 5x faster than comparable frontier models

Model, Full Access
  • NVIDIA released Nemotron 3 Ultra, a 550B-parameter open-weights reasoning model, topping the US open-weights intelligence leaderboard with a score of 47.7.
  • Uses a hybrid Mamba-Transformer + LatentMoE architecture with only 55B active parameters per token, enabling 5x higher throughput than comparable open models.
  • Scores 71.9 on SWE-Bench Verified and 94.7 on RULER at 1M token context, with the lowest hallucination rate in its comparison set.
  • Available on HuggingFace, OpenRouter, and NVIDIA NIM; weights, training data, and recipes fully open under the commercial-friendly OpenMDW license.
  • API pricing is $0.60/1M input tokens and $2.60/1M output tokens; self-hosting requires 8-10 H100s in BF16, or 4-5 with INT4 quantization.
  • Trails China's Kimi K2.6 (53.9 vs 47.7 on the Intelligence Index) and is weaker than Gemma 4 31B on the Coding Index specifically.

NVIDIA just dropped its biggest open-weights model yet: Nemotron 3 Ultra, a 550 billion parameter reasoning model that sits at the top of the US open-weights intelligence rankings. The headline number is impressive, but the more interesting story is how it gets there -- through an architectural bet that most labs haven't made at this scale.

A model built for agents, not just answers

What makes it especially notable isn't just the size -- it's what the model was specifically optimized to do: run multi-step agentic tasks reliably, reason across long contexts, and call external tools with high accuracy. It targets a specific problem: long-running agents that plan, call tools, and reason across many turns. As agents run longer, token counts grow and inference cost climbs. Nemotron 3 Ultra is designed to keep accuracy high while making that inference faster and cheaper.

Smaller models (under 70B) often lose coherence in long agentic chains -- they forget earlier context, fail on compound instructions, or make tool-calling errors that cascade into broken workflows. The 550B parameter count gives Nemotron Ultra the working memory and reasoning depth to handle tasks that take 10, 20, or 50 steps without degrading.

The architecture is the real story

The model employs a hybrid Latent Mixture-of-Experts (LatentMoE) architecture, utilizing interleaved Mamba-2 and MoE layers, along with select Attention layers. This is a meaningful departure from the standard Transformer-only design used by most frontier models. Here's what each piece does:

  • Mamba-2 layers: Mamba layers handle long sequences with sub-quadratic scaling -- meaning memory and compute grow much slower than in pure attention as context length increases.
  • Attention layers: A few Attention layers are kept for precise recall over large contexts.
  • LatentMoE routing: Tokens are projected into a smaller latent dimension for expert routing and computation, improving accuracy per byte.
  • MTP layers: The Ultra model incorporates Multi-Token Prediction (MTP) layers for faster text generation and improved quality. These enable native speculative decoding -- the model predicts multiple tokens at once, then verifies them, dramatically boosting throughput.

The Mixture-of-Experts design activates only 55B of the 550B parameters per token -- a 10x sparsity ratio. That's the key to making a 550B model feel fast: you're only ever running a fraction of it on any given token.

Trained from the ground up in FP4

Most quantized models start in full precision and get compressed afterward. Nemotron 3 Ultra takes a different path. The Ultra model is pre-trained using an NVFP4 recipe -- sharing the quantization-aware pre-training approach pioneered in the Nemotron 3 family. The majority of linear layers use NVFP4 for weights, activations, and gradients, while select layers (including latent projections, MTP layers, QKV/attention projections, and embeddings) are maintained in BF16 or MXFP8 for training stability.

The base model was pre-trained for approximately 20 trillion tokens using crawled and synthetic code, math, science, and general knowledge data. Post-training went deep: the model was further fine-tuned on synthetic code, math, science, tool calling, instruction following, structured outputs, and general knowledge data. This stage incorporated data designed to support long-range retrieval and multi-document aggregation.

The most technically interesting post-training step is what NVIDIA calls Multi-Domain On-Policy Distillation (MOPD). MOPD uses strong teacher models to guide training on the model's own generated attempts (on-policy rollouts), helping recover accuracy and improve performance across coding, math, instruction following, tool use, and agentic workflows. By distilling teacher signal onto the student's own trajectories rather than offline traces, MOPD better aligns the student's behavior with what it would actually produce at inference time, yielding stronger gains than purely off-policy distillation.

The deployment trick that fits on one node

The quantization strategy has a practical payoff beyond speed. The reduced weight footprint has a deployment benefit: the W4A16 path leaves room to fit MTP weights on a single 8-GPU H100 node. An FP8 checkpoint could not, without spanning two nodes. That's a meaningful infrastructure difference -- one node vs. two is the difference between a manageable deployment and a much more complex one.

On Blackwell it runs with native FP4 math. On Hopper it runs as W4A16, since Hopper lacks native FP4 tensor cores. So the same checkpoint works across GPU generations, with Blackwell getting the full speed benefit.

Where it stands on benchmarks

The numbers are strong across the board for agentic and reasoning tasks:

  • 71.9 on SWE-Bench Verified -- a real-world software engineering benchmark where the model must fix bugs in GitHub repositories
  • 90.0 on PinchBench and 56.0 on ProfBench (Search) on agentic tasks
  • 94.7 on RULER at 1 million tokens -- measuring whether the model can actually use its massive context window
  • The highest non-hallucination score in its comparison set on AA-Omniscience at 78.7
  • 47.7 on the Artificial Analysis Intelligence Index -- the top score among US open-weights models, ahead of Gemma 4 31B (39.2) and gpt-oss-120b (33.3)

Nemotron 3 Ultra is also built for efficiency. In experiments on SWE-bench and Terminal Bench 2.0, it completed benchmarks using fewer total tokens and fewer tokens per turn than comparable models. This lowers the cost for agentic tasks by up to 30%.

There are real limits though. It's less differentiated on pure creative writing or general knowledge tasks where other models match or exceed it. And on the Coding Index specifically, Gemma 4 31B scores about 1 point higher despite being a fraction of the size. The Chinese open-weights frontier -- led by Kimi K2.6 at 53.9 on the Intelligence Index -- still leads by a meaningful margin.

Speed that changes the agentic math

Nemotron 3 Ultra achieves 5x higher throughput compared to other open models in its class, enabling long-running agents to complete tasks faster and more efficiently. The throughput comparison against competitors on GB200 hardware is striking: on an 8K input / 64K output setting at NVFP4 on GB200, Nemotron 3 Ultra reaches 5.9x the throughput of GLM-5.1 and 4.8x faster than Kimi-K2.6. For agentic pipelines that make dozens of model calls per task, this compounds quickly into real cost and latency savings.

How to access it

Nemotron 3 Ultra is available on HuggingFace, OpenRouter, ModelScope, and NVIDIA NIM. The weights are fully open under the OpenMDW license, which permits commercial use. NVIDIA released pre-trained, post-trained, and quantized checkpoints, plus the training datasets and recipes.

Self-hosting is possible but demanding. A full-precision (BF16) deployment of a 550B model requires approximately 1TB of GPU memory -- roughly 8-10 H100 80GB GPUs. INT4 quantization can bring it down to 4-5 high-end GPUs. NVIDIA's NIM platform handles optimization automatically and offers cloud-hosted inference for teams that don't want to manage their own hardware.

Pricing via API sits at $0.60 per 1M input tokens (with a 67% cache discount to $0.20) and $2.60 per 1M output tokens. That's slightly above average for the class, but the efficiency gains on agentic tasks can offset the per-token cost.

What this changes

According to NVIDIA, it outperforms models with over a trillion total parameters on agent-focused benchmarks, runs roughly 5x faster than comparable open-weight alternatives, and costs about 30% less to operate. That's the core claim: you don't need a trillion-parameter model to get trillion-parameter-level agentic performance.

Early adopters confirmed include Accenture, CrowdStrike, Palantir, and Perplexity, suggesting real enterprise traction. The model's full openness -- weights, training data, and recipes all released -- also means teams can fine-tune it on proprietary data, something impossible with closed API models of comparable capability. For the first time, a frontier-quality model optimized for autonomous AI workloads is available without vendor lock-in, and at inference costs that don't punish you for running it at scale.

The gap with the Chinese open-weights frontier is real and worth watching. But for teams building agents on US infrastructure with commercial licensing requirements, Nemotron 3 Ultra just became the most capable option available.