PyTorch's Triton 3.7 Lets Developers Load Custom GPU Compilers Without Forking

For years, anyone who wanted to push Triton beyond its defaults faced the same painful choice: maintain a fork. Fork Triton, add your custom compiler passes, your hardware-specific memory ops, your specialized dialects, and then spend the next several months fighting merge conflicts every time upstream moved. The PyTorch-Triton 3.7 release changes that with the Triton Plugin Extensions system, a framework for dynamically loading custom compiler passes, dialects, and DSL extensions into upstream Triton at runtime, without forking or recompiling.

The fork tax was real

Writing high-performance GPU kernels often requires going beyond what the default Triton compiler pipeline provides. Custom optimization passes, hardware-specific intrinsics, and specialized memory management patterns are essential for squeezing out the last drops of performance on production workloads. Until now, enabling these capabilities meant maintaining a fork of Triton, and said forks come with real costs.

Forks fall behind upstream. Every update risks merge conflicts, broken APIs, and subtle behavioral changes. Teams that pin to a forked version miss upstream bug fixes, new hardware support, and community improvements. The maintenance burden compounds, and the fork becomes a bottleneck. The plugin system is a direct answer to this.

How the plugin system works

The framework lets you develop Triton compiler extensions that extend functionality without modifying the core Triton codebase. Extensions are built as shared libraries that can be dynamically loaded by Triton at runtime. You point Triton at your plugin via a single environment variable, TRITON_PLUGIN_PATHS, and the extensions are immediately available, no recompilation of Triton required.

The system provides three levels of extensibility:

• Custom transformation passes: single passes inserted at arbitrary points in the compilation pipeline, with no associated dialect needed.
• Custom MLIR dialects and conversion passes: separately compiled dialects loaded into Triton, with plugin passes that rewrite standard Triton IR into custom dialect ops for specialized lowering. MLIR (Multi-Level Intermediate Representation) is the compiler IR framework Triton uses internally to represent and transform GPU programs.
• Custom top-level DSL ops: new Python-level syntax and semantics enabling entirely new programming abstractions without altering Triton itself.

Plugins can also be toggled on and off at the kernel level. A compiler hook set in kernel code activates a custom pipeline for all kernels called after the hook is set, until it is unset. The 3.7 release notes confirm plugin hooks and out-of-tree dialects are now officially supported, with added support for out-of-tree TTIR/TTGIR passes and Triton Dialect Plugins.

TLX: the first major plugin, now built-in

As the first major consumer of this system, Meta's Triton Language Extensions (TLX) are now enabled out of the box, bringing persistent GEMM kernels and fine-grained hardware control to stock Triton with performance that matches or exceeds vendor libraries on both NVIDIA H100 and AMD MI350.

TLX gives kernel authors direct control over shared memory allocation, data movement, and instruction scheduling. These are the capabilities that matter for writing persistent kernels (kernels that stay resident on the GPU across multiple tiles of work, hiding memory latency through overlapped compute and data movement). The core TLX operations include:

• tlx.local_alloc: allocate shared memory buffers for software pipelining
• tlx.async_load / tlx.async_load_commit_group / tlx.async_load_wait_group: initiate and synchronize asynchronous loads from global to shared memory
• tlx.async_dot / tlx.async_dot_wait: asynchronous matrix multiply-accumulate and its barrier
• tlx.local_store / tlx.local_load: explicit shared memory read/write

The utlx package (µTLX) provides most of Meta's TLX functionality without modifying the Triton fork. It includes local memory operations, custom passes like PingPong and PruneUnusedBarriers, the TLX dialect, conversion patterns, and a Python DSL. Previously, using TLX required building from Meta's experimental Triton fork. Now it ships as a standalone Python package.

Getting started is three commands:

git clone https://github.com/triton-lang/triton && cd triton
TRITON_EXT_ENABLED=ON pip install -e . --no-build-isolation && cd ..
pip install triton-utlx

Then in Python:

import os, sysconfig
dist_packages = sysconfig.get_paths()["purelib"]
os.environ["TRITON_PLUGIN_PATHS"] = os.path.join(dist_packages, "utlx_plugin", "libutlx.so")
import triton
import triton.language as tl
import utlx_plugin as tlx  # TLX ops now available in your kernels

The numbers: matching and beating vendor libraries

The headline claim is that the plugin-loaded path introduces zero overhead compared to the compiled-in fork, and the benchmarks back it up. On NVIDIA H100, Triton + TLX matches or exceeds cuBLAS on the large, compute-bound matrix shapes that dominate LLM inference workloads:

On AMD MI350, the gains are more consistent and larger. Since the plugin system produces identical codegen to the compiled-in fork, these results apply to both paths. Triton + TLX delivers 12-15% higher throughput than rocBLAS across all tested matrix sizes, from 256×256×256 all the way to 2048×2048×2048.

The team also validated the plugin on a real production-style workload: a GPU Mode competition kernel involving five projection GEMMs feeding a batched matmul, wrapped in layer norms, sigmoid gates, and permutations. The PyTorch + torch.compile baseline ran at 19.2ms. With the TLX plugin loaded into stock Triton via TRITON_PLUGIN_PATHS, the final submission ran at 12.0ms, a 1.61x speedup over the cuBLAS + torch.compile baseline, beating all other SOTA implementations on H100.

Cross-hardware by design

One of the more interesting aspects of TLX is that the same programming model works across hardware vendors. On Hopper GPUs, TLX maps to hardware-native TMA (Tensor Memory Accelerator) async loads and WGMMA (Warp Group Matrix Multiply-Accumulate) instructions. On AMD MI350, the same buffer management pattern applies, but data movement goes through registers rather than async hardware units. The plugin handles the lowering difference transparently.

The key implementation idea is additive integration. TLX is implemented as an extension to Triton's frontend, IR builder, and lowering passes rather than as a separate compiler. This lets TLX reuse Triton's programming model and optimization pipeline, while introducing new compiler-visible constructs only where Triton's original abstraction is too coarse.

What this opens up

The plugin system is not just about TLX. The triton-ext repository is designed as a collection of out-of-tree extensions for the Triton compiler, including passes, dialects, backends, and language extensions. The roadmap includes:

• Custom backends: dynamically loaded out-of-tree backends for Intel, CPU, and other targets without modifying Triton's build system
• triton-distributed: distributed computing primitives as a plugin
• Custom profiling tools: runtime-loadable versions of Proton and ConSan for user-specific performance analysis
• Specialized ops: 2:4 structured sparsity, custom layout conversions, and target-specific warp specialization

The broader implication is a shift in how the Triton ecosystem works. Hardware vendors, research labs, and individual teams can now ship compiler extensions as Python packages on PyPI, without waiting for upstream merges or maintaining divergent forks. Extensions are enabled by default in Triton releases 3.7 and beyond. The official PyTorch blog post includes Colab notebooks and standalone scripts for both H100 and MI350 to get started immediately.