Cursor's Mixture-of-Kittens Open-Source Kernel Trains AI 2.37x Faster
Cursor open-sources Mixture-of-Kittens, a production MoE megakernel that delivers 2.37x faster throughput and 1.41x end-to-end training speedup over DeepEP on GB300 NVL72s.
- Cursor open-sources Mixture-of-Kittens (MoK), a production MoE training megakernel for NVIDIA GB300 NVL72 racks.
- MoK is up to 2.37x faster than the best public baseline (HybridEP+Megatron) on MXFP8 forward passes.
- In production on 512 GPUs, MoK raised end-to-end training throughput by 1.41x over the previous DeepEP-based stack.
- Key innovations: pull-based dispatch for better NVLink saturation, tunable minibatch granularity, ring token buffers to eliminate CPU-GPU sync, and full megakernel fusion.
- Fully deterministic by design, making it suitable for on-policy RL post-training and reproducible ablations.
- Supports BF16 and MXFP8 precision; targets DeepSeek-V3-style MoE architectures used by Kimi, Qwen, GLM, and DSV models.
Mixture-of-Kittens (MoK) is Cursor's new open-source CUDA megakernel for training Mixture-of-Experts (MoE) models on NVIDIA GB300 NVL72 racks. Already running in production to train Composer, Cursor's agentic coding model, across tens of thousands of GPUs, it benchmarks up to 2.37x faster on MXFP8 forward passes and delivered a 1.41x end-to-end training throughput gain over Cursor's previous DeepEP-based stack.
Where the time was actually going
In large MoE models, each input token is routed to a small subset of "experts" (specialized feed-forward sub-networks) that may live on different GPUs. Tokens travel across the network before and after computation in a process called dispatch and combine. Depending on the workload, the MoE layer can consume more than half of total training time. The compute portion had already been optimized; the bottleneck had shifted to inter-GPU communication.
Earlier Cursor work, including custom MXFP8 and NVFP4 training kernels and a "warp decode" approach for MoE inference, addressed only the compute side and left inter-GPU communication to separate systems. In production, communication had become the limiting factor and forced a ground-up redesign.
Moving to GB300 NVL72s added two new constraints. An NVL72 is a multi-node rack within a single NVLink domain, enabling fine-grained overlap of computation and communication across all 72 GPUs. But the integrated Grace CPUs are slow relative to the GPUs, so GPU streams quickly caught up to CPU-side work, leaving the GPU idle. Any solution had to minimize CPU involvement aggressively.
What MoK actually does
MoK fuses all MoE communication and computation into a single persistent kernel. There are no kernel launch boundaries separating dispatch, expert computation, and combine. Everything runs inside one kernel, with different groups of Streaming Multiprocessors (SMs) assigned to different tasks and coordinating through shared counters. The design is also fully deterministic: the same input produces bitwise-identical output regardless of hardware scheduling.