NVIDIA's Nemotron 3.5 Lightning Cuts Agent Costs 58% Running 4x Faster

Most of the compute in a long-running AI agent is not spent on planning. It is spent on the grind: tool calls, result validation, subagent delegation, and routine formatting. Frontier reasoning models are overkill for this work, and routing every execution step through one adds unnecessary cost and latency. NVIDIA's answer is Nemotron 3.5 Lightning, a new open model built specifically for that execution layer.

30 billion parameters, 3 billion at a time

Built for specialized tasks within larger multi-agent systems, Nemotron 3.5 Lightning is a 30-billion-parameter mixture-of-experts model designed to help create smarter and more efficient agentic applications. The key number is not 30B though. In a Mixture-of-Experts (MoE) architecture, a learned router sends each token to only a small subset of specialized sub-networks called "experts." The rest of the model sits idle. The model delivers up to 4x faster output speed, leading to 30% faster agentic task completion compared with other models in its class.

Modern agentic systems increasingly operate as systems of models, with different models specialized for different tasks. NVIDIA Nemotron open models are designed for this architecture. A frontier reasoning model such as Nemotron 3 Ultra or GPT-5.6 may plan and orchestrate a workflow, while smaller specialized models like Nemotron 3.5 Lightning can perform targeted tasks such as code review, tool use, security alert monitoring, and answering billing questions.

What is actually under the hood

Speed without accuracy is useless for agents. Lightning's design stacks several techniques to win on both axes simultaneously:

• Speculative decoding: The model was pretrained with multi-token prediction (MTP) baked in, meaning it can draft several tokens at once and verify them in parallel rather than generating one token at a time. NVIDIA also ships two dedicated draft models alongside it: DSpark (optimized for DGX Spark and low-concurrency workloads) and DFlash (for broader inference scenarios).
• Harness-optimized training: The model was trained specifically against popular agent harnesses like OpenClaw and Hermes Agent, so it makes more accurate tool calls and produces outputs that fit agent pipelines without extra post-processing.
• NVFP4 quantization: Lightning ships with both BF16 and NVFP4 checkpoints. NVFP4 is NVIDIA's 4-bit floating point format that runs on Blackwell, Hopper, and Ampere GPUs, keeping the model small enough to fit on a DGX Spark desktop workstation while preserving accuracy.

This release follows Nemotron 3 Nano and reflects NVIDIA's commitment to continually improving open models for greater accuracy and speed. Alongside the model weights, NVIDIA is releasing Nemotron-RL-Agentic-Terminal-Pivot, the open agentic reinforcement learning dataset used to train Lightning's coding agent capabilities.

The benchmark that actually matters here

Standard LLM benchmarks measure single-turn accuracy. Agents care about throughput over thousands of sequential tasks. PinchBench was designed for exactly this. The model delivers up to 4x faster output speed, leading to 30% faster agentic task completion compared with other models in its class. On PinchBench specifically, Lightning reaches 86% accuracy while completing 10,000 tasks 35% faster than Qwen3.6 35B at similar accuracy levels.

NVIDIA also benchmarked Lightning on the Artificial Analysis Intelligence Index, which aggregates nine evaluations across agentic tasks, coding, scientific reasoning, and general intelligence. Lightning sits on the Pareto frontier of that index, meaning no model of similar size achieves both higher accuracy and higher speed simultaneously.

Built to be customized

The model ships fully open under the OpenMDW-1.1 license, with weights, training data, and recipes all available. Because it's open and customizable, Nemotron 3.5 Lightning can be easily post-trained with NVIDIA NeMo on an organization's own domain data, tools and workflows to improve accuracy for specialized tasks. Several enterprises have already done this:

• CrowdStrike for cybersecurity alert monitoring
• Harvey with Trajectory for legal services
• CodeRabbit with Baseten for code review workflows
• Lila Sciences for physical and life sciences reasoning
• Fastino Labs for finance and healthcare, reporting leading accuracies in those domains

Fine-tuning options include LoRA and full SFT via NeMo Automodel, as well as reinforcement learning through NeMo RL and NeMo Gym. Small MoE models fine-tune faster and cheaper than large dense models, which is a practical advantage for teams that want domain-specific performance without massive GPU budgets.

The routing layer that ties it all together

Alongside Lightning, NVIDIA is releasing NeMo Switchyard, an open-source model routing library. When deployed, NeMo Switchyard can intelligently direct each request to the most capable and suitable model for the job without requiring developers to rewrite their applications. The idea is that complex planning routes up to a frontier model, while routine execution routes down to Lightning, so token budgets are spent where they create value.

The cost numbers from early partners are notable:

• Ramp matched frontier model performance while cutting costs by 58% and runtime by 33%
• LangChain achieved 74% lower cost in 145 multi-turn deep agent tasks by routing only 7% of calls to a frontier model
• Cognition reduced mean cost by 28% while maintaining near-frontier performance on FrontierCode Main
• Boomi achieved 100% domain-routing accuracy and reduced later-turn latency by 21%

NVIDIA is also releasing NeMo Switchyard as an open source library for smart routing inside popular agent tools. Enterprises can use it to build a router based on their specific needs.

Where it runs and how to get started

Lightning is designed to run anywhere on the NVIDIA stack. Together, Nemotron 3.5 Lightning and NeMo Switchyard deliver greater control over how AI is deployed, where it runs and how efficiently it operates across PCs, workstations, data centers and the cloud. Supported local hardware includes NVIDIA RTX PCs, DGX Spark, DGX Station, and Jetson edge devices. It also runs on standard tools like Ollama, LM Studio, llama.cpp, and Unsloth.

For cloud deployment, the model is available as an NVIDIA NIM microservice on build.nvidia.com, on Hugging Face, ModelScope, and OpenRouter (including a free tier). Hosted inference is available through Fireworks AI, Together AI, DeepInfra, CoreWeave, Baseten, and others. The official cookbook includes deployment guides for vLLM, SGLang, and TensorRT-LLM. The model is free to use for commercial purposes under the NVIDIA Nemotron Open Model License.

The real bet here is architectural: that production AI systems will increasingly be multi-model pipelines, not single-model deployments. Lightning is designed to be the workhorse of those pipelines, handling the volume so frontier models can focus on the decisions that actually require them.