--- title: "NVIDIA's Nemotron 3 Ultra Runs 5x Faster Than Open Rivals at 550B Parameters" subtitle: "NVIDIA ships Nemotron 3 Ultra, a fully open 550B MoE model that runs 5x faster than comparable frontier models and cuts agentic task costs by 30%" url: "https://alphasignal.ai/news/nvidia-s-nemotron-3-ultra-runs-5x-faster-than-open-rivals-at-550b-parameters" author: "AlphaSignal Newsroom" author_url: "https://alphasignal.ai/editorial-team" company: "NVIDIA AI" company_url: "https://alphasignal.ai/company/nvidiaai" published: "2026-06-04T13:06:18.000Z" updated: "2026-06-16T20:23:40.411Z" type: "News" topics: - "Open Source" - "Agents" subtopics: - "Agent Frameworks" upvotes: 4295 read_time: "7 min" source_url: "https://developer.nvidia.com/blog/nvidia-nemotron-3-ultra-powers-faster-more-efficient-reasoning-for-long-running-agents/?linkId=100000424980294" comments: 0 --- # NVIDIA's Nemotron 3 Ultra Runs 5x Faster Than Open Rivals at 550B Parameters > NVIDIA ships Nemotron 3 Ultra, a fully open 550B MoE model that runs 5x faster than comparable frontier models and cuts agentic task costs by 30% The open-weight frontier just got a serious upgrade. NVIDIA has shipped [Nemotron 3 Ultra](https://developer.nvidia.com/blog/nvidia-nemotron-3-ultra-powers-faster-more-efficient-reasoning-for-long-running-agents/), a 550-billion-parameter model built from the ground up for one specific problem: making long-running AI agents actually practical to deploy at scale. The model is fully open -- weights, training data, synthetic datasets, and post-training recipes are all available now on [Hugging Face](https://huggingface.co/nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-NVFP4). The timing matters. The two models currently at the top of the frontier are proprietary, API-only, and priced accordingly. The closest open-weights competitor requires roughly 862GB of VRAM to run -- effectively a dedicated GPU cluster. Nemotron 3 Ultra is NVIDIA's answer to both constraints: intelligence approaching the frontier, open weights, and an architecture engineered for throughput rather than just accuracy. ## The agent problem nobody talks about Single-turn chat is easy. Agents are not. In multi-agent workflows, token counts grow quickly. Agents plan, call tools, invoke sub-agents, receive information, and then pass history, outputs, and reasoning steps back into the model continuously. As tasks run longer, this constant communication increases costs and the risk of goal drift. Nemotron 3 Ultra is a 550B-parameter Mixture-of-Experts model with 55B active parameters, built for frontier reasoning and orchestration in agentic systems. Within any agent workflow, most calls are routine, but a critical subset demands deeper reasoning. The model is designed to handle those hard calls: sustaining architectural decisions across long coding sessions, synthesizing contradictory evidence across hundreds of research sources, or verifying chip designs across thousands of constraints. ## Speed and cost that actually move the needle Nemotron 3 Ultra achieves 5x higher throughput compared to other open models in its class, enabling long-running agents to complete tasks faster and more efficiently. That speed advantage compounds over time -- an agent that can do more reasoning cycles within the same time budget simply gets more done. In experiments on SWE-bench and Terminal Bench 2.0, it completed benchmarks using fewer total tokens and fewer tokens per turn than comparable models. This lowers the cost for agentic tasks by up to 30%. That efficiency gain comes from the model being smarter about when to reason deeply versus when to answer directly -- a behavior baked in through post-training, not just architectural tricks. ## Five innovations under the hood The performance numbers come from a stack of architectural choices that each solve a specific bottleneck for agentic workloads: - **Hybrid Mamba-Transformer:** Mamba layers improve sequence efficiency for long-context workloads, while Transformer layers preserve precise recall when agents need to retrieve specific facts from large context windows. Mamba is a state-space model architecture -- think of it as a more memory-efficient alternative to attention for processing long sequences. - **NVFP4 quantization:** The same NVFP4 checkpoint runs on NVIDIA Hopper, Blackwell, and Ampere GPUs. NVFP4 also delivers up to 5x higher throughput per GPU at the same interactivity compared to BF16 on Blackwell. One checkpoint, any modern NVIDIA GPU. - **LatentMoE:** The model utilizes the LatentMoE architecture, where tokens are projected into a smaller latent dimension for expert routing and computation, improving accuracy per byte. Standard MoE routes tokens to experts at full dimension; LatentMoE compresses first, routes second, which is more efficient. - **Multi-Token Prediction (MTP):** Multi-token prediction helps reduce generation time by predicting multiple future tokens in a single forward pass, improving throughput for long outputs and multi-turn workflows. - **Agent harness post-training:** Ultra is optimized for agent-led open harnesses, not just single-turn chat, and is designed to work within workflows where agents plan, call tools, read observations, delegate to sub-agents, validate outputs, and recover from errors across many turns. ## A new training trick: Multi-Teacher On-Policy Distillation The most interesting methodological innovation is what NVIDIA calls Multi-Teacher On-Policy Distillation (MOPD). The idea: instead of training from a single teacher model's outputs, you train from more than ten specialized teacher models simultaneously -- each an expert in its own domain. During MOPD, the student model generates rollouts across domains and receives dense reward signals from the corresponding teacher models. To maximize efficiency, MOPD runs asynchronously, with student rollout generation, teacher scoring, and student optimization fully pipelined. The key insight is that the student generates its own attempts first (on-policy), then gets graded by the relevant teacher. This is fundamentally different from just copying a teacher's outputs offline. MOPD is also iterative. After producing an MOPD-trained checkpoint, new rounds of teacher training are initialized from the updated student model, and the improvements are merged into the next MOPD stage. This co-evolution between students and teachers enables continuous capability improvement and progressively stronger specialization across domains. ## A four-stage training pipeline The full training process has four distinct phases: 1. **Pre-training:** The base model was pre-trained for approximately 20 trillion tokens using crawled and synthetic code, math, science, and general knowledge data. 2. **Supervised Fine-Tuning:** Fine-tuned on synthetic code, math, science, tool calling, instruction following, and long-range retrieval data. 3. **Reinforcement Learning:** The model underwent multi-environment reinforcement learning using asynchronous GRPO (Group Relative Policy Optimization) across math, code, science, instruction following, multi-step tool use, multi-turn conversations, and structured output environments. GRPO is a variant of RL that compares groups of model outputs against each other rather than using a separate reward model. 4. **MOPD:** The final stage described above, where domain-specialized teachers push the model's reasoning further across all task types. On top of the base 20T token pre-training foundation, Nemotron 3 Ultra adds 212B new tokens targeting three high-value domain gaps: 4B tokens of synthetic legal data, 35B tokens of synthesized Wiki-based data, and 173B refreshed GitHub tokens through September 2025. ## How it benchmarks The benchmark picture is nuanced. Ultra is not the best at everything, but it holds its own while being dramatically faster than models that beat it on specific tasks. | Benchmark | Nemotron 3 Ultra (550B) | GLM 5.1 (744B) | Kimi K2.6 (1T) | Qwen3.5 (397B) | | --- | --- | --- | --- | --- | | Agent Productivity (PinchBench) | **91%** | 84% | 91% | 89% | | Instruction Following (IFBench) | **82%** | 77% | 74% | 78% | | Long Context (RULER @1M) | **95%** | N/A (max 256K) | N/A (max 256K) | 90% | | Coding (Terminal-Bench 2.0) | 54% | 64% | **67%** | 53% | | Long-horizon Planning (EnterpriseOps-Gym) | 33% | **40%** | 29% | 30% | It tops US open-weight rankings yet trails China's Kimi K2.6 by six intelligence-index points -- while running several times faster. For most production agent workloads, that speed-accuracy tradeoff is a clear win. On SWE-bench Verified, it scores between 65% and 70.4% across different agent frameworks, showing consistent performance regardless of which scaffolding you deploy. ## Where it struggles The benchmarks reveal some honest gaps. Long-horizon planning (EnterpriseOps-Gym) is a relative weak spot at 33%, where GLM 5.1 scores 40%. Raw coding performance on Terminal-Bench 2.0 also lags behind Kimi K2.6 by 13 points. Given the 550B parameter scale, full fine-tuning is expensive -- LoRA and GRPO recipes are provided, but you will need serious hardware to run them. There is also a deployment caveat worth flagging. The 1M token context is what the model can do architecturally. The context a given provider actually serves can be lower -- endpoints frequently cap served context well below the architectural ceiling for cost and latency reasons. Confirm the served limit on whichever provider you deploy against before designing a long-context pipeline around it. ## The ecosystem play NVIDIA is not just shipping a model -- it is shipping a stack. Two companion models launched alongside Ultra: - **Nemotron 3.5 Content Safety:** A 4B guardrail model covering 23 safety categories and 12 languages, usable as an inference-time filter or an evaluation judge. - **Nemotron 3.5 ASR:** A multilingual streaming speech recognition model supporting 40+ languages with sub-100ms latency, using the same architecture that already powers voice input in GitHub Copilot CLI for over 20 million developers. Nemotron models are moving to OpenMDW-1.1, the Linux Foundation's permissive license purpose-built for open AI model distributions. This gives developers and enterprises clearer terms for using, modifying, redistributing, and deploying Nemotron models, while reducing the licensing ambiguity that can slow evaluation and adoption of open models. ## Getting started The NVFP4 checkpoint is the recommended starting point. Minimum hardware is 4x B200 for single-node deployment, or 8x H100 for Hopper-generation setups. You can serve it with vLLM, SGLang, or TRT-LLM. Here is the minimal vLLM quickstart: ``` pip install vllm vllm serve ``` ## Sources - [X post by @nvidiaai](https://x.com/nvidiaai/status/2062521325076299981)