PyTorch's ExecuTorch Runs AI Models on Browser GPUs up to 10x Faster

Developers shipping PyTorch models in web applications have typically converted them to formats such as ONNX, GGUF, or TFLite, then maintained a separate runtime and conversion pipeline. PyTorch’s new WebGPU backend for ExecuTorch packages an exported model as a .pte file that can run on the GPU in supported versions of Chrome and Safari.

PyTorch’s WebGPU support reduces deployment work across operating systems while moving inference into the browser sandbox. Applications can keep prompts, images, audio, and model outputs on the user’s device, provided the surrounding application does not transmit that data. Local execution can also remove per-request inference server costs.

One artifact crosses browser GPUs

The developer workflow starts with torch.export, followed by ExecuTorch lowering and packaging. The resulting model artifact can travel with the application’s JavaScript and WebAssembly assets through ordinary static hosting or a CDN.

1. Export the PyTorch model with torch.export.
2. Partition and lower supported graph regions for WebGPU.
3. Package the graph and weights in a .pte file.
4. Load the artifact with the ExecuTorch WebAssembly runtime.
5. Run inference through the browser’s WebGPU implementation.

WebGPU exposes compute shaders, storage buffers, shared memory, synchronization barriers, and command buffers to web applications. ExecuTorch compiles its C++ runtime to WebAssembly with Emscripten, while emdawnwebgpu maps the runtime’s GPU calls to the browser API. PyTorch describes the architecture and export process in its release post, with implementation details available in the backend source.

From graph partitions to shaders

During export, WebGPUPartitioner identifies graph regions supported by the backend and lowers their operations to WGSL, WebGPU’s shading language. Unsupported operations require compatible ExecuTorch kernels or additional backend coverage, making operator support a key check before deployment.

At runtime, the backend allocates GPU buffers, builds compute pipelines, and records shader dispatches in command buffers. This work stays within the browser process and uses the GPU selected by the browser’s WebGPU adapter.

Dynamic input shapes use buffers sized for the configured maximum shape. The runtime recalculates workgroup counts for each dispatch, allowing workloads such as batched prompt prefill to change shape without recompiling every kernel.

Each browser translates WGSL for the host graphics stack, including Metal Shading Language on macOS, SPIR-V for Vulkan platforms, and HLSL for Direct3D 12 on Windows. That translation lets the same .pte model target several operating systems, subject to browser, hardware, and operator support.

Where the GPU pulls ahead

PyTorch benchmarked quantized Llama 3.2 1B and Qwen3-0.6B in Chrome on an Apple M1 Pro. The comparison used ExecuTorch’s XNNPACK-backed WebAssembly CPU path as the baseline.

The GPU path used 4-bit groupwise weights with a group size of 64 and FP32 activations. XNNPACK used 4-bit weights with dynamically quantized 8-bit activations. Those numerical formats differ, so the results compare deployable runtime paths rather than identical GPU and CPU kernels. They also cover one Apple system and should be validated on the devices an application intends to support.

Coverage extends beyond chat

PyTorch reports successful runs across language, vision, speech, multimodal, and training workloads, each of which exercises a different mix of operators and memory-access patterns.

• Language models: Llama 3.2 1B, Qwen2.5-0.5B, Qwen3-0.6B, and Gemma 3 1B, including quantized configurations.
• Vision-language: SmolVLM for image questions and captioning.
• Computer vision: SAM 2 Hiera Tiny for segmentation, Depth Anything V2 for depth estimation, D-FINE for object detection, Real-ESRGAN for super-resolution, and MODNet for background removal.
• Speech: Whisper base for automatic speech recognition and Phonon-31M-EN for text-to-speech.
• Browser training: LoRA and DiReFT fine-tuning for Llama 3.2 1B, Qwen2.5-0.5B, and Qwen3-0.6B.

Each demonstrated model has a Hugging Face repository containing the .pte artifact, model information, and an export recipe. Those recipes provide a reproducible path from the original checkpoint to the browser package.

Workloads that fit local execution

Local GPU inference suits applications that handle sensitive data, need offline behavior, or would otherwise send frequent requests to an inference service. Likely uses include:

• Chat assistants that keep conversation history on the device.
• Transcription tools that process microphone audio locally.
• Image editors that perform segmentation, upscaling, or background removal.
• Camera applications that estimate depth or detect objects without uploading frames.
• Personalized models fine-tuned with LoRA or DiReFT in the browser.

Production limits remain

Production teams still need to test the backend against their complete model graph and browser matrix. The current release carries several practical constraints:

• Operator coverage: Models containing unsupported operations may need graph changes, fallback kernels, or new backend implementations.
• Data types: Precision and quantization support remain narrower than mature native GPU stacks.
• Memory: Weights, activations, and maximum-shape buffers must fit within the GPU resources exposed by the browser.
• Downloads: Large model files increase startup time and bandwidth use, making compact and quantized models more practical.
• Compatibility: WebGPU availability varies by browser version, operating system, device, and enterprise policy.
• Stability: The backend remains in early development and requires application-specific testing before production use.

Developers can report missing operators and runtime problems through the ExecuTorch repository. The WebGPU Playground provides browser demos for chat, Whisper transcription, SAM segmentation, and depth estimation.