NVIDIA Releases Nemotron 3.5 Lightning With 1M-Token Context on One GPU
NVIDIA's new 30B hybrid Mamba-Transformer MoE activates just 3B parameters, runs on a single H100, and handles up to 1M tokens of context.
- Nemotron 3.5 Lightning is a 30B hybrid Mamba-2 plus MoE plus Attention model with 3B active parameters.
- Context length reaches 1M tokens on GB200 or 8x H100, and 256K on a single H100.
- Pre-trained on over 20T tokens using an NVFP4 recipe, with GRPO reinforcement learning for post-training.
- Ships with three speculative decoding methods: DSpark, DFlash, and embedded MTP heads.
- Scores 81.94 on MMLU Pro and 51.56 on SWE-bench Verified, trailing Qwen 3.6 35B A3B on most benchmarks.
- Released under OpenMDW 1.1, commercial use allowed, with full training dataset collections published.
NVIDIA has released Nemotron 3.5 Lightning 30B A3B, a 30-billion-parameter open-weight model that activates about 3 billion parameters for each token. According to the model card, it combines Mamba-2 state-space layers, selective Transformer attention, sparse experts, and Multi-Token Prediction heads. The design targets long-context customization and efficient serving on hardware ranging from 80GB data-center GPUs to Blackwell systems.
| Specification | Details |
|---|---|
| Total parameters | 30 billion |
| Active parameters | About 3 billion per token |
| Architecture | Hybrid Mamba-2, attention, and Mixture-of-Experts layers |
| Maximum context | 1 million tokens |
| Single 80GB GPU context | Up to 256,000 tokens |
| Reference checkpoint | BF16 |
| Languages | English, Spanish, French, German, Italian, Japanese, and programming languages |
| License | OpenMDW 1.1, with commercial use permitted under its terms |
How 3B active parameters work
Mamba-2 layers process sequences with work that grows roughly linearly with sequence length. Standard full self-attention compares token pairs, producing quadratic attention work as the sequence expands. Lightning retains selected attention layers to provide direct token-to-token lookup for retrieval while using state-space layers across most of the network.
Mixture-of-Experts routing sends each token through a subset of the model’s expert weights. That routing reduces per-token arithmetic, but a single-device server still loads the full 30-billion-parameter weight set. Memory sizing therefore follows the complete checkpoint, while generation cost depends largely on the active path, routing overhead, and attention layers.
Multi-Token Prediction heads learn to predict several future positions from the same hidden state. Those heads provide an additional training signal and can serve as an integrated draft mechanism during speculative decoding.
Long context, bounded by memory
NVIDIA advertises a 1-million-token context window, with the usable limit determined by accelerator memory, checkpoint precision, batch size, and serving configuration. The model card lists these deployment targets:
| Hardware | Reported context | Configuration |
|---|---|---|
| 1 H100 80GB or A100 80GB | 256,000 tokens |
This story is for Pro members
You've reached the end of the free preview. Upgrade to AlphaSignal Pro to read the full article - and everything else behind the paywall.
Comments
