--- title: "NVIDIA's Nemotron 3 Ultra Hits 6x Faster Inference With 550B Open Model" subtitle: "NVIDIA's 550B-parameter open model lands free on OpenCode, combining a 1M-token context window with up to 6x higher throughput than rival open LLMs" url: "https://alphasignal.ai/news/nvidia-s-nemotron-3-ultra-hits-6x-faster-inference-with-550b-open-model" author: "AlphaSignal Newsroom" author_url: "https://alphasignal.ai/editorial-team" company: "OpenCode" company_url: "https://alphasignal.ai/company/opencode" published: "2026-06-04T16:21:46.000Z" updated: "2026-06-16T20:21:32.869Z" type: "News" topics: - "Llms" - "Open Source" subtopics: - "Long Context" - "Small Models" upvotes: 4446 read_time: "7 min" source_url: "https://research.nvidia.com/labs/nemotron/Nemotron-3-Ultra/" comments: 0 --- # NVIDIA's Nemotron 3 Ultra Hits 6x Faster Inference With 550B Open Model > NVIDIA's 550B-parameter open model lands free on OpenCode, combining a 1M-token context window with up to 6x higher throughput than rival open LLMs NVIDIA just dropped its biggest open model yet, and it's free to use right now. [Nemotron 3 Ultra](https://research.nvidia.com/labs/nemotron/Nemotron-3-Ultra/) is a 550 billion total parameter model that activates only 55 billion of those parameters per token, and it's now available at no cost through [OpenCode](https://opencode.ai), the open-source terminal coding agent. The tweet that lit up the community was simple: "text · 1M context · fully open source." With 250k+ views, the response makes clear that the combination of scale, context length, and zero price tag hit a nerve. This is not a minor model update. Nemotron 3 Ultra is the flagship of the Nemotron 3 family, announced at GTC San Jose 2026, and it represents NVIDIA's most serious push yet into the foundation model space. The architecture, training pipeline, and open release strategy all point to a deliberate attempt to challenge proprietary frontier models on their own turf. ## A new kind of architecture under the hood Nemotron 3 Ultra is NVIDIA's largest open model: 550B total parameters with up to 55B active per token via a hybrid Mamba-Transformer mixture-of-experts (MoE) architecture. The key word is *hybrid*. Nemotron 3 Ultra is not a standard transformer MoE. The architecture combines three components in a single hybrid stack: Mamba-2 SSM layers, MoE feed-forward layers, and standard attention layers. Each of these three components does a different job. The MoE routing keeps compute low: the model has 550B total parameters across experts, but only 55B activate on any given forward pass. You need 550 GB of VRAM to hold all expert weights (at FP8), but the compute per token is equivalent to a 55B dense model. You pay for storage, not for compute. The Mamba-2 layers handle the long context: SSM layers maintain a recurrent state that scales linearly with sequence length rather than quadratically like attention. That linear scaling is what makes a 1M-token context window practical rather than theoretical. Then there is LatentMoE, NVIDIA's proprietary twist on the standard MoE router. LatentMoE compresses tokens into a low-rank latent space before routing, enabling 4x as many expert specialists for the same inference cost. More experts for the same compute budget means more specialization without paying more at inference time. Finally, Multi-Token Prediction (MTP) predicts multiple future tokens in a single forward pass, improving chain-of-thought coherence and enabling built-in speculative decoding at inference time -- no separate draft model required. ## How it was trained NVIDIA pre-trained Nemotron 3 Ultra on 20 trillion text tokens, then extended the context length to 1M tokens, and post-trained using Supervised Fine Tuning (SFT), Reinforcement Learning (RL), and Multi-teacher On-Policy Distillation (MOPD). The pretraining itself was done in NVFP4, a 4-bit floating point format. To NVIDIA's knowledge, this is the largest-scale demonstration of stable and accurate NVFP4 training to date. The data pipeline is unusually transparent. NVIDIA released the training datasets alongside the model weights, including a two-phase curriculum: 15T tokens biased for diversity, then 5T biased for quality, plus 173B new GitHub tokens with a September 30, 2025 cutoff. The post-training step uses MOPD, which means distilling 10+ specialized teachers into one student. Each teacher is an expert in a different domain; the student learns to combine their outputs. ## The throughput story is the real headline NVIDIA's benchmark claim is striking: Nemotron 3 Ultra achieves up to ~6x higher inference throughput compared to state-of-the-art publicly available LLMs while attaining on-par accuracy. The specific numbers: Nemotron 3 Ultra achieves 5.9x, 4.8x, and 1.6x higher inference throughput compared to GLM-5.1-754B-A40B, Kimi-K2.6-1T-A32B, and Qwen-3.5-397B-17B respectively on the 8k token input / 64k token output setting. Early third-party data from Artificial Analysis reported more than 300 output tokens per second on a pre-release DeepInfra endpoint. For developers, this speed changes the interaction model entirely. If a model is slow, you fire and forget. If it is fast, you stay in the loop, ask follow-ups, and let an agentic harness keep pushing. Nemotron 3 Ultra is particularly strong at multi-step reasoning and planning, with high-throughput inference designed for high-volume agent pipelines. On accuracy, the benchmarks are competitive: the model recorded 71.9 on SWE-Bench Verified, surpassing Qwen-3.5 (69.9), suggesting that AI has evolved from a mere code writer to an autonomous engineering agent. On long-context retrieval, it recorded 94.7 on RULER at 1M tokens, clearly pulling away from Qwen-3.5 (90.1) and DeepSeek-v4-Flash (87.7). ## What it's actually good for The model is optimized for complex agentic workflows, long-context reasoning, and high-stakes analytical workloads. It is intended for developers designing AI agent systems, chatbots, RAG systems, and other AI-powered applications. It is also suitable for complex instruction-following tasks and long-context reasoning over very large documents and codebases. The practical sweet spots, based on early testing and CodeRabbit's internal benchmark: - Code review pipelines where comments can be verified, filtered, and retried - Integration test generation requiring broad codebase context - Repository research tasks that need to scan many files at once - Agentic workflows where a harness can keep the model moving until a task is complete - Deep research tasks that require reasoning across multiple documents simultaneously ## The catch: it needs structure CodeRabbit ran Nemotron 3 Ultra through 105 internal evaluation problems and found that quality was close to their baseline -- but reliability was not. The model had a high retry rate, averaging 36.5 retries per run compared to 0.3 for the baseline. About 66% of those retries were caused by the model stopping before producing the required output marker. Retrying without changing the prompt often worked, which suggests the capability is there, but first-attempt structured output is not yet stable. Ultra needs structure. If you are using it for coding or developer automation, do not treat it as a free-form chat model and hope it infers the workflow. The practical guidance from early testers: - Use explicit prompts with concrete acceptance criteria - Validate required output markers before accepting a response - Add retry logic for structured-output tasks - Ask for tests explicitly -- the model does not always generate them unprompted - Prefer it for high-throughput workflows where several attempts are acceptable ## Fully open, fully free to start NVIDIA released everything: pre-trained, post-trained, and quantized checkpoints along with the datasets used for training. The checkpoints available on Hugging Face include the post-trained BF16 model, the NVFP4-quantized version, the base model, and even the GenRM reward model used during RLHF. The [NVIDIA Nemotron Developer Repository](https://github.com/NVIDIA-NeMo/Nemotron) on GitHub contains training recipes and usage cookbooks. To use it free right now via OpenCode, the config is straightforward: ``` ollama launch opencode --model nemotron-3-ultra:cloud ``` Or configure it directly via the OpenCode API: ``` { "providers": { "opencode": { "models": [{ "id": "nemotron-3-ultra-free" }], "baseUrl": "https://opencode.ai/zen/v1" } } } ``` It is also available on [OpenRouter](https://openrouter.ai/nvidia/nemotron-3-ultra-550b-a55b:free) for free, and via NVIDIA's own build.nvidia.com API. The full technical details are in the [official technical report](https://research.nvidia.com/labs/nemotron/files/NVIDIA-Nemotron-3-Ultra-Technical-Report.pdf). ## Why this matters beyond the benchmarks The open-weight AI landscape has been dominated by a few players -- Meta's Llama series, Mistral, and a handful of Chinese labs. NVIDIA entering with a model of this scale, releasing all weights and training data, and making it free through popular developer tooling is a meaningful shift. The company's hardware business gives it a structural incentive to push open models: more capable open models drive demand for GPUs. Nemotron 3 Ultra is a 550B open MoE (55B active) using a hybrid Mamba-Attention design for long-running agents. NVIDIA reports up to ~6x higher inference throughput than comparable open LLMs at on-par accuracy. That combination -- frontier-class accuracy, dramatically higher throughput, 1M-token context, and a zero-dollar entry point -- is a hard combination to ignore for any team building agentic systems. The question is no longer whether open models can compete with proprietary ones on quality. It is whether they can compete on reliability and developer experience. Ultra is a strong answer to the first part of that question. ## Sources - [X post by @opencode](https://x.com/opencode/status/2062570516586573998) - [NVIDIA Nemotron 3 Ultra - Official Page](https://research.nvidia.com/labs/nemotron/Nemotron-3-Ultra/) - [NVIDIA Nemotron 3 Ultra - Official Docs (Base Model)](https://docs.nvidia.com/nemotron/nightly/usage-cookbook/Nemotron-3-Ultra-Base/README.html) - [Nemotron 3 Ultra makes the case for fast, open coding models - CodeRabbit](https://www.coderabbit.ai/blog/nemotron-3-ultra-release) - [Nemotron 3 Ultra on OpenRouter (free)](https://openrouter.ai/nvidia/nemotron-3-ultra-550b-a55b:free) - [NVIDIA Nemotron 3 Ultra Technical Report (PDF)](https://research.nvidia.com/labs/nemotron/files/NVIDIA-Nemotron-3-Ultra-Technical-Report.pdf)