--- title: "NVIDIA Ships Nemotron 3 Ultra, the Most Capable Open US Model at 550B" subtitle: "NVIDIA's Nemotron 3 Ultra is the most capable open-weight model from a US lab, delivering 5x faster inference than rivals at 30% lower cost for agentic workloads" url: "https://alphasignal.ai/news/nvidia-ships-nemotron-3-ultra-the-most-capable-open-us-model-at-550b" author: "AlphaSignal Newsroom" author_url: "https://alphasignal.ai/editorial-team" company: "Artificial Analysis" company_url: "https://alphasignal.ai/company/artificialanlys" published: "2026-06-04T18:12:22.000Z" updated: "2026-06-16T20:20:23.684Z" type: "News" topics: - "Llms" - "Benchmarks" subtopics: - "Vision Language" upvotes: 192 read_time: "8 min" source_url: "https://artificialanalysis.ai/articles/nvidia-nemotron-3-ultra-released" comments: 0 --- # NVIDIA Ships Nemotron 3 Ultra, the Most Capable Open US Model at 550B > NVIDIA's Nemotron 3 Ultra is the most capable open-weight model from a US lab, delivering 5x faster inference than rivals at 30% lower cost for agentic workloads NVIDIA just shipped [Nemotron 3 Ultra](https://developer.nvidia.com/blog/nvidia-nemotron-3-ultra-powers-faster-more-efficient-reasoning-for-long-running-agents/), a 550-billion-parameter open-weight reasoning model built from the ground up for long-running AI agents. It is now the most capable open model released by any US lab, scoring 47.7 on the Artificial Analysis Intelligence Index -- well ahead of the next best American open models, Gemma 4 31B (39.2) and gpt-oss-120b (33.3). The model is available today, weights and all, under the permissive OpenMDW-1.1 license from the Linux Foundation. The headline number is not the benchmark score. It is the speed. Through BlackBox AI, Nemotron 3 Ultra is served at over 400 output tokens per second -- slightly faster than the typical serving speed of gpt-oss-120b despite being more than 4x larger, and with significantly greater intelligence. For agentic workloads where a model gets called thousands of times per workflow, that throughput gap changes the economics of the entire system. ## Built for agents, not chatbots The design goal here is explicit. NVIDIA did not build this to win single-turn chatbot comparisons. It built it to orchestrate agents that plan, call tools, delegate to sub-agents, read observations, and recover from errors across hundreds of turns, while burning fewer tokens than comparable open models. NVIDIA claims up to 5x higher throughput and up to 30% lower cost to complete agentic tasks than other open models in its class. The model fits into a tiered system NVIDIA is pushing hard: Nano handles edge and high-volume execution, Super handles mid-tier workhorse tasks, and Ultra sits at the top as the frontier reasoning and orchestration layer. Because Ultra ships under a permissive license, its outputs can legally be used to post-train the smaller models below it -- something the terms of most closed frontier models prohibit. Reasoning is also configurable. The model supports three modes: full thinking, medium effort, and reasoning off, all toggled through the chat template. You can also set a hard token ceiling on the thinking trace so the model stops deliberating once it hits the cap -- a practical control for agent builders watching token spend. ## The architecture doing the heavy lifting Nemotron 3 Ultra is NVIDIA's largest open model: 550B total parameters with up to 55B active per token via a hybrid Mamba-Transformer mixture-of-experts (MoE) architecture. That ratio -- 550B total, 55B active -- is the core of the efficiency story. It carries the knowledge capacity of a massive model while only paying the inference cost of a much smaller one. Several architectural innovations work together to make this possible: - **Hybrid Mamba-Transformer:** Mamba-2 layers handle long sequences far more efficiently than attention, which is what makes the 1-million-token context window practical without cost exploding. - **LatentMoE:** Tokens are compressed into a low-rank latent space before routing, enabling 4x as many expert specialists for the same inference cost. - **Multi-Token Prediction (MTP):** The model predicts multiple future tokens in a single forward pass, improving chain-of-thought coherence and enabling built-in speculative decoding at inference time. - **NVFP4 precision:** The same checkpoint runs across Hopper, Blackwell, and Ampere GPUs -- one build everywhere, with NVIDIA claiming up to 5x higher throughput per GPU vs BF16 on Blackwell. ![Scatter plot showing Nemotron 3 Ultra's position in the most attractive quadrant for intelligence vs output speed](https://cdn.sanity.io/images/6vfeftx9/articles/f1897c436c4c1ff3445e72786fe2d7ca03934425-4640x2112.png?w=1200&auto=format) ## A novel training recipe: MOPD The most technically interesting piece of the training pipeline is a method NVIDIA calls Multi-Teacher On-Policy Distillation (MOPD). In MOPD, the Ultra model learns from more than 10 specialized teacher models -- each trained on a specific domain like coding, math, search, or tool calling -- while generating its own attempts during training. A coding rollout gets graded by the coding teacher, a math rollout by the math teacher, and so on, feeding back dense reward signals across domains simultaneously. The teacher lineup includes Qwen3 variants, DeepSeek V3, R1, and V4-Pro, GPT-OSS-120B, GLM-5, Mistral variants, and phi-4. The whole loop runs asynchronously -- student rollout generation, teacher scoring, and student optimization are fully pipelined so no stage stalls waiting on another. After each MOPD checkpoint, NVIDIA initializes new rounds of teacher training from the updated student, then merges those improvements into the next stage. Student and teachers co-evolve. Earlier training stages are more conventional: supervised fine-tuning with a May 2026 cutoff focuses on code, math, science, tool calling, and instruction following. Pre-training ran on roughly 20 trillion tokens. NVIDIA has released the MOPD recipe through NeMo-RL so others can reproduce it. ## Where it wins, where it doesn't Independent benchmarks from Artificial Analysis paint a clear picture. Nemotron 3 Ultra scores 47.7 on the Artificial Analysis Intelligence Index, well ahead of the next strongest US open weights models, but behind the Chinese-led open weights frontier (Kimi K2.6 at 53.9). The standout strength is long context. Ultra holds 95% on Ruler at 1 million tokens, while both GLM 5.1 and Kimi K2.6 top out at 256K and cannot run the test at all. On agent productivity (PinchBench), it ties Kimi K2.6 at 91% despite being roughly half the parameter count. On instruction following (IFBench), it leads the field at 82%. The gaps are real too: - **Raw coding:** On Terminal-Bench 2.0, Ultra scores 54%, behind GLM 5.1 at 64% and Kimi K2.6 at 67%. - **Long-horizon planning:** On EnterpriseOps-Gym, Ultra scores 33%, behind GLM 5.1 at 40%. - **Peak intelligence:** At 48 on the AA Intelligence Index, it trails Kimi K2.6 (54) and closed models like Opus 4.8 (61). - **Graduate-level physics:** CritPt benchmark performance sits at just 3%, unchanged from Nemotron 3 Super. The honest read is that Nemotron 3 Ultra wins on the axes NVIDIA optimized for -- agent productivity, instruction following, long context, speed, and cost -- and trails on the axes it traded away. ## The latency-vs-performance tradeoff, quantified Artificial Analysis ran a novel evaluation on Terminal-Bench v2.1 that captures something standard benchmarks miss: how models perform under time pressure. They gave each model four different turn budgets (10, 20, 50, and 100 turns) and traced the curve of task completion rate versus wall-clock time per task. Nemotron 3 Ultra is the fastest across all turn limits and sits on the Pareto frontier for performance versus time per task for this evaluation. In other words, at every level of effort you're willing to spend, it completes more tasks per minute than any peer. This matters because real agent deployments don't run in unlimited-turn mode. Systems impose turn budgets to control cost and latency. A model that is both fast and capable under those constraints is more useful in production than one that only shines with unlimited compute. ## Getting your hands on it Availability at launch is unusually broad. You can access it through: - **Weights:** [Hugging Face](https://huggingface.co/nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16) in BF16 and NVFP4 variants - **Managed APIs:** Perplexity Pro, OpenRouter, build.nvidia.com, DeepInfra, Together AI, Fireworks AI, CoreWeave, Google Cloud, AWS JumpStart, and more - **Agent harnesses:** BlackBox AI, OpenHands, CrewAI, LangChain Deep Agents, Cline, Kilo Code, and Factory AI - **Self-hosted:** NVIDIA NIM microservice with deployment recipes for vLLM, SGLang, and TensorRT-LLM For self-hosting, the FP8 checkpoint is approximately 253 GB. Current support for Nemotron 3 Ultra is limited to NVIDIA Blackwell architecture (including B200/B300 and GB200/GB300), while support for NVIDIA Hopper systems is planned but not currently available for the 550B variant. The 253B instruct variant fits on a single 8xH100 node. On pricing, Nemotron Ultra 253B API inference starts from $2.00/M input tokens and $6.00/M output tokens through NVIDIA's NIM, with third-party providers like Nebius offering rates as low as $0.72 per million blended tokens. ``` # Quick start with vLLM (FP8 checkpoint) vllm serve /models/nemotron-ultra-fp8 \ --quantization modelopt \ --tensor-parallel-size 8 \ --trust-remote-code \ --gpu-memory-utilization 0.90 \ --max-model-len 32768 \ --port 8000 # Exposes an OpenAI-compatible /v1/chat/completions endpoint ``` ## The bigger picture: open vs. closed, US vs. China Nemotron 3 Ultra lands in a specific geopolitical and competitive context. American companies -- OpenAI, Anthropic, Google -- keep their best systems behind APIs. It's a structural gap, not a talent gap. US labs compete on closed, subscription-based products. Chinese labs like Moonshot AI, DeepSeek, and Alibaba's Qwen team invest heavily in open-weight releases. NVIDIA is now the most credible US lab willing to compete in that open-weight arena. The model also signals a broader thesis shift. The Nemotron 3 models introduce a breakthrough hybrid latent mixture-of-experts architecture that helps developers build and deploy reliable multi-agent systems at scale. As organizations shift from single-model chatbots to collaborative multi-agent AI systems, developers face mounting challenges, including communication overhead, context drift, and high inference costs. NVIDIA's answer is a tiered system of models where a strong, fast orchestrator hands routine work to cheap executors -- and where the orchestrator's outputs can legally train those executors because the license allows it. Early enterprise adoption backs the thesis. Glean, an enterprise AI platform, added Nemotron 3 Ultra citing it delivering 91% of frontier LLM completeness at open-model cost. Their own agentic search model, Waldo, is post-trained on Nemotron 3 Nano and reportedly runs with 50% lower latency and 25% fewer tokens. That is exactly the system-of-models pattern playing out in production. Whether efficiency beats raw intelligence as the primary axis of competition in agentic AI is the question Nemotron 3 Ultra is betting on. ## Sources - [X post by @artificialanlys](https://x.com/artificialanlys/status/2062598349757567359) - [NVIDIA Nemotron 3 Ultra released: fast, intelligent, and open](https://artificialanalysis.ai/articles/nvidia-nemotron-3-ultra-released) - [NVIDIA Nemotron 3 Ultra Just Released - Fello AI](https://felloai.com/nvidia-nemotron-3-ultra/) - [NVIDIA Nemotron 3 Ultra Base Model - NVIDIA Docs](https://docs.nvidia.com/nemotron/nightly/usage-cookbook/Nemotron-3-Ultra-Base/README.html) - [nvidia/Llama-3_1-Nemotron-Ultra-253B-v1 - Hugging Face](https://huggingface.co/nvidia/Llama-3_1-Nemotron-Ultra-253B-v1) - [Deploy Nemotron Ultra 253B on GPU Cloud - Spheron Blog](https://www.spheron.network/blog/deploy-nemotron-ultra-253b-gpu-cloud/)