NVIDIA's Nemotron 3.5 Super VL Runs 5x Faster on Agentic Tasks
NVIDIA's new open 120B mixture-of-experts model fuses text, image, document, and video reasoning into one agent-ready backbone with 12B active parameters.
- NVIDIA released Nemotron 3.5 Super VL, an open 120B MoE multimodal model with 12B active parameters.
- Single model handles text, image, document, and video plus reasoning and tool use for agents.
- Up to 5x higher throughput than peer open models and 4x faster coding task completion on B200.
- Partners report 40x lower inference cost, 10x lower latency, and 40% accuracy gains after post-training.
- Uses LiLiCorr speculative decoding, NVFP4 with QAD, multi-token prediction, and Mamba cache reuse.
- Weights, data, and recipes available on Hugging Face, with NIM, vLLM, and Dynamo deployment paths.
NVIDIA has released Nemotron 3.5 Super VL, an open-weight multimodal model for agents that work across text, images, documents, video, and external tools. Its mixture-of-experts architecture contains 120 billion parameters but activates 12 billion for each token, reducing per-token computation while retaining the capacity of a larger model.
For developers, the practical appeal lies in consolidating perception, reasoning, and tool use within one backbone. That design can remove handoffs between a text model and a separate vision model during workflows such as reading scanned invoices, interpreting charts, searching the web, and querying databases.
What NVIDIA shipped
| Area | Details |
|---|---|
| Architecture | 120B-parameter mixture of experts with 12B parameters active per token |
| Inputs | Text, images, documents, and video |
| Checkpoints | Base BF16, post-trained, and NVFP4-quantized variants |
| Access | Model card and hosted inference through OpenRouter |
| Deployment | NVIDIA NIM, vLLM, SGLang, Ollama, and third-party inference providers |
| Primary use cases | Document processing, coding, visual reasoning, tool use, and long-running enterprise agents |
One backbone across the agent loop
Super VL combines instruction following, reasoning, visual understanding, and tool calling. A document agent can therefore inspect a scanned invoice, read a chart on another page, preserve the surrounding context, and produce a database call without serializing intermediate results through a second model.
Within NVIDIA’s Nemotron lineup, smaller Lightning models target high-volume text workloads, while the larger Nemotron 3 Ultra handles more demanding planning and orchestration. Super VL occupies the middle tier, with an emphasis on specialized multimodal workflows that require several reasoning and tool-use steps.
The numbers and their limits
NVIDIA reports up to five times higher single-GPU throughput on agentic tasks than other open models in the same size category. A production comparison still requires matched hardware, precision, context length, batch size, output length, and concurrency because each variable can materially change throughput.
Partner deployments
- Scale AI and Skydance: Scale AI post-trained the model for automated vendor credit processing at Skydance Corporation. The deployment is projected to reduce inference cost by 40 times and latency by five times.
- Ramp: Ramp is using the model to convert receipt images into structured JSON containing merchant, date, and payment fields. The company expects up to 50% higher throughput and 10 times lower latency.
- AI21 and Alice: AI21 post-trained the model for Alice’s threat-intelligence and vulnerability-analysis workflows, reporting a 40% improvement in accuracy.
These partner figures cover different tasks, baselines, hardware configurations, and post-training methods, so they describe individual deployments rather than a common head-to-head benchmark.
On public evaluations cited by NVIDIA, Super VL outscored the larger Nemotron 3 Ultra on Hermes Bench and LangChain agent tests. It recorded 80.8% on the CharXiv chart-reasoning benchmark, compared with 63% for Nemotron 3 Nano Omni, and reached 84% when allowed to interpret charts with Python code.
Distillation plus faster decoding
NVIDIA derived Super VL from Nemotron 3 Ultra rather than training it entirely from scratch. The distillation process transferred Ultra’s coding capabilities and used a training objective called SkewReverse loss, which NVIDIA says improves coding accuracy.
Post-training used multi-teacher on-policy distillation. During that process, Super VL generates responses and specialized teacher models evaluate them for tasks including video reasoning, object tracking, vision-tool use, and alignment. The student model then learns from feedback tailored to each capability.
- LiLiCorr speculative decoding: A lightweight network scores the compatibility of draft tokens in one pass, after which a lower-cost sequential step selects the final draft. NVIDIA reports up to 2.3 times faster token generation on DGX Spark than decoding without speculation.
- NVFP4 quantization-aware distillation: The Blackwell-focused recipe uses 4-bit weights and activations. Distillation helps the quantized model preserve the teacher’s behavior, while the key-value cache can also use NVFP4 instead of FP8 when accuracy targets permit.
- Multi-token prediction: An internal drafter proposes several tokens for the main model to verify in one forward pass, reducing the number of expensive verification steps.
- Context reuse: The serving stack preserves attention caches and Mamba recurrent states. Additional Mamba checkpoints reduce the amount of prior input that must be replayed during speculative decoding.
References to attention caches and Mamba recurrent states indicate a hybrid Transformer and Mamba serving path. Transformer attention retains token-to-token relationships, while Mamba modules maintain compact recurrent state that can reduce the cost of processing long or repeatedly reused context.
Post-training drives specialization
The release includes base and post-trained checkpoints because NVIDIA expects organizations to adapt the model to their own documents, tools, policies, and output formats. NeMo AutoModel supports supervised fine-tuning and low-rank adaptation, while reinforcement-learning workflows can train behavior inside agent environments.
NVIDIA is also publishing datasets for agentic tool use, financial reasoning, instruction following, long-context translation, fairness, and multimodal training. The release includes 45 vision-language datasets intended to support domain-specific adaptation and evaluation.
An accompanying Indic-language dataset contains roughly 2.8 trillion tokens across 22 Indian languages. NVIDIA developed the collection with IIT Madras and its AI4Bharat lab.
Repeated context changes the serving math
NVIDIA’s Dynamo benchmark used 16 B200 GPUs and a target generation rate of 50 tokens per second for each user. With key-value-aware routing, session affinity, and cache offload, the system served up to eight times as many users and reduced time to first token by 85% relative to the baseline in NVIDIA’s test.
The reported gain applies to workloads with substantial shared prefixes, a common pattern for agents that repeatedly send the same system prompt, tool schemas, and recent conversation history. Key-value-aware routing sends a request to a worker that already holds relevant attention state, while session affinity keeps later turns on the same worker and cache offload frees GPU memory.
What to test before deployment
- End-to-end quality: Evaluate scanned documents, charts, screenshots, tool calls, and structured outputs as complete trajectories rather than isolated prompts.
- Serving performance: Measure time to first token, output tokens per second, concurrency, and tail latency with production-length prompts and outputs.
- Prefix reuse: Reproduce the actual overlap in system prompts, tool definitions, and conversation history before estimating Dynamo’s benefit.
- Memory requirements: Account for the full 120B-parameter checkpoint even though 12B parameters activate per token. Compare BF16 and NVFP4 deployments on supported hardware.
- Post-training cost: Include dataset preparation, evaluation, reinforcement learning, and regression testing in the total deployment budget.
- License and data terms: Review the checkpoint license, dataset conditions, and redistribution requirements on the model card.
Nemotron 3.5 Super VL gives teams an open-weight option for consolidating multimodal perception, reasoning, and tool use in specialized agents. Its practical advantage will depend on workload-specific post-training, hardware, and repeated-context patterns, making direct evaluation against existing models and APIs the decisive deployment test.
Comments
