NVIDIA's Nemotron 3 Ultra Beats Trillion-Parameter Models at 5x the Speed
NVIDIA's Nemotron 3 Ultra is a fully open 550B-parameter reasoning model that tops the US open-weights leaderboard while running 5x faster than comparable frontier models
- NVIDIA released Nemotron 3 Ultra, a 550B-parameter open-weights reasoning model, topping the US open-weights intelligence leaderboard with a score of 47.7.
- Uses a hybrid Mamba-Transformer + LatentMoE architecture with only 55B active parameters per token, enabling 5x higher throughput than comparable open models.
- Scores 71.9 on SWE-Bench Verified and 94.7 on RULER at 1M token context, with the lowest hallucination rate in its comparison set.
- Available on HuggingFace, OpenRouter, and NVIDIA NIM; weights, training data, and recipes fully open under the commercial-friendly OpenMDW license.
- API pricing is $0.60/1M input tokens and $2.60/1M output tokens; self-hosting requires 8-10 H100s in BF16, or 4-5 with INT4 quantization.
- Trails China's Kimi K2.6 (53.9 vs 47.7 on the Intelligence Index) and is weaker than Gemma 4 31B on the Coding Index specifically.
NVIDIA just dropped its biggest open-weights model yet: Nemotron 3 Ultra, a 550 billion parameter reasoning model that sits at the top of the US open-weights intelligence rankings. The headline number is impressive, but the more interesting story is how it gets there -- through an architectural bet that most labs haven't made at this scale.
A model built for agents, not just answers
What makes it especially notable isn't just the size -- it's what the model was specifically optimized to do: run multi-step agentic tasks reliably, reason across long contexts, and call external tools with high accuracy. It targets a specific problem: long-running agents that plan, call tools, and reason across many turns. As agents run longer, token counts grow and inference cost climbs. Nemotron 3 Ultra is designed to keep accuracy high while making that inference faster and cheaper.
Smaller models (under 70B) often lose coherence in long agentic chains -- they forget earlier context, fail on compound instructions, or make tool-calling errors that cascade into broken workflows. The 550B parameter count gives Nemotron Ultra the working memory and reasoning depth to handle tasks that take 10, 20, or 50 steps without degrading.
The architecture is the real story
The model employs a hybrid Latent Mixture-of-Experts (LatentMoE) architecture, utilizing interleaved Mamba-2 and MoE layers, along with select Attention layers. This is a meaningful departure from the standard Transformer-only design used by most frontier models. Here's what each piece does:
- Mamba-2 layers: Mamba layers handle long sequences with sub-quadratic scaling -- meaning memory and compute grow much slower than in pure attention as context length increases.
- Attention layers: A few Attention layers are kept for precise recall over large contexts.
- LatentMoE routing: Tokens are projected into a smaller latent dimension for expert routing and computation, improving accuracy per byte.
- MTP layers: The Ultra model incorporates Multi-Token Prediction (MTP) layers for faster text generation and improved quality. These enable native speculative decoding -- the model predicts multiple tokens at once, then verifies them, dramatically boosting throughput.