Skip to main content
Back

NVIDIA Quietly Shrinks Kimi K2.6 to Run on Blackwell GPUs 3.5x Smaller

NVIDIA published an NVFP4-quantized checkpoint of Moonshot's trillion-parameter Kimi K2.6, shrinking it for Blackwell GPUs with near-zero accuracy loss.

Model, Full Access
  • NVIDIA released Kimi-K2.6-NVFP4, an NVFP4 quantization of Moonshot's 1T-parameter MoE.
  • Native FP4 execution on Blackwell tensor cores, roughly 1.8x smaller than FP8 in memory.
  • Accuracy essentially matches the INT4 baseline across GPQA, SciCode, MMMU Pro, and IFBench.
  • Ships as a vLLM-ready checkpoint with Kimi's tool-call and reasoning parsers.
  • Base Kimi K2.6 supports 256K context, 300-agent swarms, and 13-hour coding runs.
  • Licensed under NVIDIA Open Model License plus Moonshot's Modified MIT for commercial use.

NVIDIA has quietly dropped a pre-quantized build of Moonshot AI's Kimi K2.6 on Hugging Face, converting the trillion-parameter agentic model into NVFP4 format so it can run on Blackwell GPUs with a much smaller memory footprint. The checkpoint, nvidia/Kimi-K2.6-NVFP4, is a straight repackaging of Moonshot's weights through NVIDIA's Model Optimizer toolchain, and it slots directly into vLLM without any code changes on the user side.

What actually shipped

Kimi K2.6 is Moonshot's flagship agentic system: an open-weight trillion-parameter Mixture-of-Experts model that leads headline agentic and coding benchmarks against GPT-5.4 and Claude Opus 4.6. It activates 32 billion parameters per token, routes through 384 experts (8 selected plus 1 shared) across 61 layers with 64 attention heads and Multi-head Latent Attention, handles a 256K context window, and accepts native image and video input.

NVIDIA's contribution is a serving-ready quantization rather than a new model. The checkpoint was produced by converting the weights and activations from INT4 to BF16 to NVFP4, ready for inference in vLLM. Only the linear operators inside the MoE transformer blocks are quantized, and the target hardware is explicitly NVIDIA Blackwell (B200 and family).