Cursor's Mixture-of-Kittens Open-Source Kernel Trains AI 2.37x Faster
Cursor open-sources Mixture-of-Kittens, a production MoE megakernel that delivers 2.37x faster throughput and 1.41x end-to-end training speedup over DeepEP on GB300 NVL72s.
- Cursor open-sources Mixture-of-Kittens (MoK), a production MoE training megakernel for NVIDIA GB300 NVL72 racks.
- MoK is up to 2.37x faster than the best public baseline (HybridEP+Megatron) on MXFP8 forward passes.
- In production on 512 GPUs, MoK raised end-to-end training throughput by 1.41x over the previous DeepEP-based stack.
- Key innovations: pull-based dispatch for better NVLink saturation, tunable minibatch granularity, ring token buffers to eliminate CPU-GPU sync, and full megakernel fusion.
- Fully deterministic by design, making it suitable for on-policy RL post-training and reproducible ablations.
- Supports BF16 and MXFP8 precision; targets DeepSeek-V3-style MoE architectures used by Kimi, Qwen, GLM, and DSV models.
Mixture-of-Kittens (MoK) is Cursor's new open-source CUDA megakernel for training Mixture-of-Experts (MoE) models on NVIDIA GB300 NVL72 racks. It is already running in production, powering the training of Composer, Cursor's agentic coding model, across tens of thousands of GPUs. Benchmarked against every major public baseline, it is up to 2.37x faster on MXFP8 forward passes and delivered a 1.41x end-to-end training throughput gain over Cursor's previous DeepEP-based stack.
The bottleneck nobody talks about enough
In large MoE models, each input token is routed to a small subset of "experts" (specialized feed-forward sub-networks) that may live on different GPUs. This means tokens have to travel across the network before and after computation, a process called dispatch and combine. Depending on the workload and training configuration, the MoE layer can consume more than half of end-to-end training time. The compute part of that layer had already been optimized; the bottleneck had shifted to inter-GPU communication.
Earlier work at Cursor, including custom MXFP8 and NVFP4 training kernels and a "warp decode" approach for MoE inference, optimized only the compute portion and assumed inter-GPU communication would be handled separately. In production workloads, communication had become the limiting factor. That realization forced a ground-up redesign.
The move to GB300 NVL72s added two new constraints. An NVL72 is a multi-node rack within a single NVLink domain, enabling fast, fine-grained overlap of computation and communication across all 72 GPUs. But the integrated Grace CPUs tend to be slow relative to the GPUs, so GPU streams easily caught up to CPU-side work, causing the GPU to be completely idle during that time. Any solution had to aggressively minimize CPU involvement.
What MoK actually does
MoK fuses all MoE communication and computation into a single kernel, is fully deterministic, and achieves state-of-the-art performance against publicly available implementations. The "megakernel" design means there are no kernel launch boundaries separating dispatch, expert computation, and combine. Everything runs inside one persistent kernel, with different groups of Streaming Multiprocessors (SMs) assigned to different tasks and coordinating through shared counters.