---
title: "DeepSeek's DeepGEMM Fuses Entire AI Expert Layers Into One GPU Operation"
subtitle: "DeepSeek's open-source CUDA kernel library gets Mega MoE, FP8×FP4 GEMM, and a fused MoE mega-kernel that overlaps NVLink comms with tensor core compute"
url: "https://alphasignal.ai/news/deepseek-s-deepgemm-fuses-entire-ai-expert-layers-into-one-gpu-operation"
author: "AlphaSignal Newsroom"
author_url: "https://alphasignal.ai/editorial-team"
company: "DeepSeek"
company_url: "https://alphasignal.ai/company/deepseek_ai"
published: "2026-06-23T10:58:46.000Z"
updated: "2026-06-23T19:55:51.472Z"
type: "Repo"
topics:
  - "Gpus"
  - "Training Infra"
subtopics:
  - "Kernels"
  - "Quantization"
upvotes: 1394
source_url: "https://github.com/deepseek-ai/DeepGEMM"
comments: 0
truncated: true
---

# DeepSeek's DeepGEMM Fuses Entire AI Expert Layers Into One GPU Operation

> DeepSeek's open-source CUDA kernel library gets Mega MoE, FP8×FP4 GEMM, and a fused MoE mega-kernel that overlaps NVLink comms with tensor core compute

[DeepGEMM](https://github.com/deepseek-ai/DeepGEMM), DeepSeek's open-source CUDA kernel library for high-performance matrix multiplication, just received its most substantial update since launch. The April 2026 release lands Mega MoE, FP8×FP4 GEMM, an FP4 lightning indexer, Programmatic Dependent Launch (PDL) support, and a faster JIT compilation path , all under the MIT license.

## What DeepGEMM actually is

DeepGEMM is a small CUDA library that takes a few well-chosen FP8 GEMM and MoE shapes seriously, hand-tunes them on top of CuTe and CUTLASS primitives, and packages them behind a JIT-compiled Python API. It is the kernel layer DeepSeek wished existed when shipping V3, R1, V3.2, and now V4 in FP8\. 

Standard cuBLAS GEMM is tuned for large square matrices. MoE expert layers use grouped GEMM with variable batch dimensions , not all experts get equal token load per step, so the effective (M, N, K) shapes change constantly. A standout feature of DeepGEMM is its use of fine-grained scaling. In the context of LLMs, GEMM operations are the primary computational bottleneck. By applying fine-grained scaling within these kernels, DeepGEMM allows for more precise control over the quantization process, which is vital when working with the limited dynamic range of 8-bit formats. 

## The April 2026 update, piece by piece

The headline additions are Mega MoE, FP8×FP4 GEMM, an FP4 Indexer, PDL, and faster JIT compilation. Here is what each one means in practice:

* **Mega MoE** , a fused mega-kernel that collapses the entire MoE forward pass into a single GPU operation. It fuses expert dispatch, the first linear layer, SwiGLU activation, the second linear layer, and expert combine , while overlapping NVLink communication with tensor core computation. Only FP8×FP4 MoE is supported, and it requires PyTorch 2.9 or higher.

_Preview only. Full text requires an AlphaSignal Pro account._

## Sources

- [deepseek-ai/DeepGEMM — GitHub](https://github.com/deepseek-ai/DeepGEMM)
- [\[Public release 26/04\] Introducing Mega MoE, FP4 Indexer and other features/fixes — PR #304](https://github.com/deepseek-ai/DeepGEMM/pull/304)
- [Add various optimizations and Mega MoE benchmarks — PR #316](https://github.com/deepseek-ai/DeepGEMM/pull/316)
- [DeepGEMM Guide: FP8 Kernels, Mega MoE, and Hopper/Blackwell — antigravity.codes](https://antigravity.codes/blog/deepgemm-guide)
- [Deploy DeepEP and DeepGEMM on GPU Cloud — Spheron Blog](https://www.spheron.network/blog/deploy-deepep-deepgemm-moe-inference-kernels-gpu-cloud/)