Google's LiteRT.js Runs AI Models in Browsers up to 60x Faster

Google just launched LiteRT.js, a JavaScript binding of its on-device inference library LiteRT that lets you run ML models entirely inside the browser -- no server, no cloud API call. It's the web-facing layer of the same runtime that powers AI on Android, iOS, and desktop, now compiled to WebAssembly and wired up to WebGPU for GPU acceleration.

If you've been using TensorFlow.js, this is the intended upgrade path. For developers with existing .tflite models, LiteRT.js makes deployment to mobile and desktop web browsers smoother than ever, serving as a powerful evolution from TensorFlow.js for executing .tflite models.

Why TensorFlow.js wasn't enough

The core problem with TensorFlow.js was always performance. Prior web AI solutions like TensorFlow.js relied on less performant JavaScript-based kernels. LiteRT.js replaces those with a native C++ runtime compiled to WebAssembly, giving you the same optimized inference path that runs on-device on mobile -- just delivered through the browser sandbox.

The conversion story is also much cleaner. The LiteRT.js conversion path goes directly from PyTorch to LiteRT. The PyTorch to TensorFlow.js conversion path is significantly more complicated, requiring you to go from PyTorch to ONNX to TensorFlow to TensorFlow.js. That multi-step chain was a real friction point, and LiteRT.js eliminates it.

Three hardware backends, one API

LiteRT.js exposes three inference backends, each targeting a different piece of hardware:

• CPU via XNNPACK: XNNPACK is Google's highly optimized library for on-device CPU acceleration, providing robust multi-thread support and a relaxed SIMD build for enhanced performance.
• GPU via ML Drift + WebGPU: ML Drift is Google's leading solution for on-device GPU acceleration. LiteRT.js leverages WebGPU to enable state-of-the-art GPU acceleration on the web.
• NPU via WebNN (experimental): For GPU and dedicated hardware scaling such as NPUs, LiteRT.js natively surfaces both the WebGPU API and the emerging WebNN API, empowering fine-grained platform-specific optimization. WebNN is currently experimental in Chrome and Edge.

The NPU path (WebNN) is the most forward-looking piece here. NPUs -- dedicated neural processing chips -- are now standard on modern laptops and phones. The WebNN API is an emerging web standard designed to provide high-performance neural network execution directly in the browser across CPUs, GPUs, and NPUs. WebNN serves as a unified graph execution API that allows frameworks to leverage platform-native backends such as Core ML on Apple, DirectML on Windows, and LiteRT/XNNPACK on Google/Android. LiteRT.js is one of the first major runtimes to wire into it.

The numbers

Across classical computer vision and audio processing models, LiteRT.js delivers significant speedups -- outperforming other web runtimes by up to 3x across both CPU and GPU inference. The gap widens considerably when you move off CPU entirely.

For demanding real-time applications like object tracking, audio transcription, or image manipulation, leveraging the GPU or NPU via WebGPU or WebNN delivers 5-60x speedup compared to standard CPU execution. Those benchmarks were run on a 2024 Apple MacBook Pro with M4 silicon, so results will vary by hardware.

What you can actually build with it

The demos Google shipped alongside the announcement show the practical range:

• Real-time object detection: YOLO models running in the browser via the official Ultralytics LiteRT export integration
• Monocular depth estimation: Depth-Anything-V2 turning a webcam feed into a live 3D point cloud via WebGPU
• 4x image upscaling: Real-ESRGAN upscaling 128x128 patches to 512x512, reassembled client-side
• Vector search: Semantic embedding search in the browser using EmbeddingGemma
• Audio transcription: Whisper-class ASR running entirely locally

These aren't toy demos. They're the kinds of tasks that previously required a backend inference server. Running them client-side means zero server costs, no round-trip latency, and user data that never leaves the device.

Getting started

Install the @litertjs/core package from npm with npm install @litertjs/core. Then loading and running a model with GPU acceleration is a handful of lines:

import { loadLiteRt, loadAndCompile, Tensor } from '@litertjs/core';
await loadLiteRt('path/to/wasm/directory/');
const model = await loadAndCompile('path/to/your/model.tflite', { accelerator: 'webgpu' });
const inputTypedArray = new Float32Array(1 * 3 * 244 * 244);
const inputTensor = new Tensor(inputTypedArray, [1, 3, 244, 244]);
const results = await model.run(inputTensor);
// Move result from GPU memory to CPU
const resultArray = (await results[0].moveTo('wasm')).toTypedArray();

LiteRT.js supports compiling from your preferred ML framework natively: PyTorch, JAX, or TensorFlow. LiteRT.js is designed to function within TensorFlow.js pipelines, and is compatible with TensorFlow.js pre- and post-processing, so the only thing you need to migrate is the model itself. That's a meaningful migration story for existing TF.js users.

Current limitations

It's not all smooth sailing. A few real constraints to know about:

• Some LiteRT models may not be supported. Errors usually fall into shape mismatch (a known GPU bug) or unsupported operations where the runtime doesn't have a mapped implementation for an operation in the model topology.
• LiteRT.js exclusively supports int32 and float32 buffer layouts at the tensor evaluation stage -- so quantized models using other dtypes need special handling.
• Raw artifact sizes can occasionally exceed the WASM memory limit in certain environments; solutions for WebGPU and WebNN are underway.
• WebNN NPU support is still experimental and requires enabling flags in Chrome or Edge.

The bigger picture

LiteRT.js is part of a broader unification effort at Google. As LiteRT.js shares a unified cross-platform stack with LiteRT, web applications automatically benefit from the latest performance upgrades, quantization improvements, and hardware optimizations developed for Android, iOS, and desktop. The web is no longer a second-class target -- it's part of the same deployment surface.

For LLM use cases, there's a companion project: LiteRT-LM's production-proven inference pipelines are now fully accessible on the web through a JavaScript API. Powered by WebGPU, LiteRT-LM delivers lightning-fast LLM routing and execution client-side, unlocking web applications that are serverless, secure, and completely privacy-preserving. The two libraries cover complementary ground -- LiteRT.js for classical and vision models, LiteRT-LM for generative.

Pretrained .tflite models are available on the LiteRT Hugging Face community, and the full source is on the LiteRT GitHub repo. LiteRT.js is available now, free, via npm.