NVIDIA's Nemotron 3.5 Lightning Cuts Agent Costs 58% Running 4x Faster
NVIDIA's open 30B MoE model with only 3B active parameters targets the high-volume execution layer of always-on AI agents, delivering 4x faster output speed than comparable models
- New open model: NVIDIA released Nemotron 3.5 Lightning, a 30B MoE model with only 3B active parameters built for high-volume agent execution.
- Speed benchmark: Delivers up to 4x faster output speed vs. similar-sized models; completes 10,000 agentic tasks 35% faster than Qwen3.6 35B at comparable accuracy.
- Fully open: Weights, training data, and recipes released under OpenMDW-1.1; free for commercial use on Hugging Face, ModelScope, and OpenRouter.
- NeMo Switchyard: A new open-source model routing library routes agent tasks to the right model automatically, with partners reporting 28-74% cost reductions.
- Customization-first: Post-train with LoRA or full SFT via NeMo; CrowdStrike, Harvey, CodeRabbit, and others have already fine-tuned it for domain-specific tasks.
- Runs anywhere: Supported on DGX Spark, RTX PCs, Jetson, Ollama, LM Studio, and major cloud providers including Google Cloud, Azure, and OCI.
Most of the compute in a long-running AI agent is not spent on planning. It is spent on the grind: tool calls, result validation, subagent delegation, and routine formatting. Frontier reasoning models are overkill for this work, and routing every execution step through one adds unnecessary cost and latency. NVIDIA's answer is Nemotron 3.5 Lightning, a new open model built specifically for that execution layer.
30 billion parameters, 3 billion at a time
Built for specialized tasks within larger multi-agent systems, Nemotron 3.5 Lightning is a 30-billion-parameter mixture-of-experts model designed to help create smarter and more efficient agentic applications. The key number is not 30B though. In a Mixture-of-Experts (MoE) architecture, a learned router sends each token to only a small subset of specialized sub-networks called "experts." The rest of the model sits idle. The model delivers up to 4x faster output speed, leading to 30% faster agentic task completion compared with other models in its class.
Modern agentic systems increasingly operate as systems of models, with different models specialized for different tasks. NVIDIA Nemotron open models are designed for this architecture. A frontier reasoning model such as Nemotron 3 Ultra or GPT-5.6 may plan and orchestrate a workflow, while smaller specialized models like Nemotron 3.5 Lightning can perform targeted tasks such as code review, tool use, security alert monitoring, and answering billing questions.
What is actually under the hood
Speed without accuracy is useless for agents. Lightning's design stacks several techniques to win on both axes simultaneously:
- Speculative decoding: The model was pretrained with multi-token prediction (MTP) baked in, meaning it can draft several tokens at once and verify them in parallel rather than generating one token at a time. NVIDIA also ships two dedicated draft models alongside it: DSpark (optimized for DGX Spark and low-concurrency workloads) and DFlash (for broader inference scenarios).