Google Launches TPU Developer Hub to Finally Make TPUs as Easy as GPUs

Google's new TPU Developer Hub centralizes tutorials, open-source recipes, and debugging tools for the full ML lifecycle on Cloud TPUs

·
·
·
Google Launches TPU Developer Hub to Finally Make TPUs as Easy as GPUs
  • Google launched the TPU Developer Hub, a centralized, code-first portal covering the full ML lifecycle on Cloud TPUs.
  • The hub features tutorials on hardware architecture, XProf debugging, Pallas custom kernels, parallelism strategies, and production inference.
  • TorchTPU, a ground-up rewrite of PyTorch-on-TPU with native eager execution, is a major highlight and will eventually replace PyTorch/XLA.
  • XProf's new Kernel Profiling suite provides sub-microsecond hardware telemetry for Pallas kernels, previously an opaque black box to profilers.
  • The hub is timed alongside Ironwood (TPU v7) and TPU 8th-gen launches, with guides for dual-chiplet architecture and large-scale pod training.
  • Content is designed to be agent-ingestion friendly, compatible with AI coding assistants pulling in best practices automatically.

Google just launched the TPU Developer Hub, a centralized learning portal designed to help teams actually get performance out of Cloud TPUs, not just provision them. The hub covers the full model lifecycle, from architecting massive pre-training clusters to squeezing latency out of production inference, with code-first guides, interactive Colabs, and open-source recipes.

The problem it solves

TPUs have long had a reputation problem: the hardware is exceptional, but the path from "I have a PyTorch model" to "it's running efficiently on a TPU pod" has been painful. Traditionally, Google only had first-class support on the JAX/XLA stack, and treated PyTorch on TPU as a second-class citizen, relying on lazy tensor graph capture through PyTorch/XLA. Scattered documentation, unfamiliar tooling, and a steep learning curve kept many teams on GPUs even when TPUs would have been the better economic choice.

In 2026, a large fraction of the world's frontier AI models runs on silicon almost nobody writes low-level code for. While the GPU ecosystem produces a torrent of CUDA kernels and Triton blog posts, Google's TPU hums along underneath Gemini, the Vertex AI stack, and a growing slice of third-party training workloads, mostly invisibly. The Developer Hub is Google's attempt to change that.

What's in the hub

The hub is described as a code-first, agent-friendly resource for ML developers, meaning the content is structured to be consumed both by humans browsing manually and by AI coding assistants pulling in context. It covers five major areas:

  • Hardware architecture: Understanding TPU design, memory hierarchy (VMEM scratchpad vs HBM), and how to pick the right infrastructure tier, from bare-metal kernels to managed Cloud TPU services.
  • Software stack: The XLA compiler, JAX AI stack, and the new TorchTPU backend. TorchTPU enables developers to migrate existing PyTorch workloads to TPU with minimal code changes.
  • Tracing and debugging: XProf tooling to gain granular visibility into workloads and pinpoint performance bottlenecks with precision.
  • Parallelism and optimization: Advanced scaling techniques including multi-chip execution models and Pallas kernels to maximize efficiency, with proven recipes for managing parallelism from basic configurations to complex distributed training setups, plus optimized strategies for advanced inference like KV cache offloading.
  • Networking and security: End-to-end guides for distributed training infrastructure, inter-chip interconnect (ICI) topology, and enterprise security practices.

TorchTPU: the PyTorch story finally makes sense

One of the most significant things the hub surfaces is TorchTPU, Google's ground-up rewrite of the PyTorch-on-TPU experience. The old PyTorch/XLA approach required developers to think in static computation graphs, use unfamiliar SPMD APIs, and deal with compilation overhead at every turn. The new core principle is simple: it should feel like PyTorch. A developer should be able to take an existing PyTorch script, change their initialization to "tpu", and run their training loop without modifying a single line of core logic.

TorchTPU establishes an "Eager First" philosophy. Instead of forcing developers into static graph compilation immediately, it is implemented using PyTorch's "PrivateUse1" interface, with no subclasses, no wrappers, just ordinary, familiar PyTorch Tensors on a TPU. Under the hood, the translation layer maps PyTorch operators directly into StableHLO, XLA's primary intermediate representation for tensor math, creating a direct connection from PyTorch into XLA's core lowering path to generate highly optimized TPU binaries.

For teams writing custom operations, TorchTPU natively supports custom kernels written in Pallas and JAX. By decorating a JAX function with @torch_tpu.pallas.custom_jax_kernel, engineers can write low-level hardware instructions that interface directly with the lowering path.

Once TorchTPU is public it will replace PyTorch/XLA. The public GitHub repo and broader ecosystem integrations with vLLM and TorchTitan are still on the 2026 roadmap.

XProf gets serious about kernel-level debugging

The hub also spotlights a major upgrade to XProf, Google's open-source ML performance profiler. The new Kernel Profiling suite addresses a long-standing blind spot: when you write a custom Pallas kernel, legacy profilers treat the entire compiled region as an opaque black box.

Customizing high-performance kernels has historically introduced a major engineering challenge: optimization blind spots. To legacy performance profilers, custom compilation paths appear as opaque execution paths, leaving developers with single, massive execution blocks in their trace captures, lacking granular visibility into what is actually occurring inside the chip's internal components.

The new Kernel Profiling suite in XProf is a low-level hardware debugging suite engineered specifically for Pallas kernel authoring and optimization on Google TPUs. By combining static compilation tracking with dynamic, sub-microsecond hardware telemetry, XProf Kernel provides the deep transparency required to optimize high-scale ML workloads.

The practical impact is measurable. Implementing triple buffering to overlap memory loads with MXU compute can reduce runtime from 125.5 microseconds to 88 microseconds, a roughly 30% performance gain validated by a drastic reduction in synchronization events. XProf also now includes Continuous Profiling Snapshots, which act as an always-on flight recorder: it operates quietly in the background with minimal system overhead (approximately 7 microseconds per packet CPU overhead), utilizing a host-side circular buffer of roughly 2GB to seamlessly retain the last ~90 seconds of performance data.

The hardware context: Ironwood and beyond

The timing of the hub is not accidental. Google has been aggressively expanding its TPU lineup. Ironwood is Google's seventh-generation TPU, specifically designed for inference. This powerful AI accelerator is built to handle the massive computational demands of thinking models like large language models and mixture of experts, scaling up to 9,216 chips and offering 42.5 Exaflops of compute power.

Today's frontier models, including Google's Gemini, Veo, Imagen, and Anthropic's Claude, train and serve on Tensor Processing Units. Anthropic's commitment to deploy over one million Ironwood chips beginning in 2026 demonstrates the architecture's production readiness, with the company planning to leverage well over a gigawatt of TPU capacity exclusively for training and serving Claude models.

The hub includes dedicated guides for training on Ironwood TPUs, including how to handle its dual-chiplet architecture. Ironwood exposes each chip as two separate "devices," one for each chiplet, and you can specify which chiplet to use for a computation. The hub's Ironwood-specific recipes cover how to navigate this when sharding large models across a pod.

Who should actually use this

The hub is most useful for teams that fall into one of these categories:

  • PyTorch teams evaluating TPUs: The TorchTPU content and migration guides are the clearest on-ramp Google has ever published for this audience.
  • JAX practitioners scaling up: The parallelism recipes, MaxText tutorials, and JAX Scaling Book cover the gap between single-device JAX and multi-pod distributed training.
  • Inference engineers: Guides on vLLM on TPU, KV cache offloading, and the Google Inference Quickstart API with benchmarking data across model sizes and hardware configs.
  • Kernel authors: The XProf Kernel Profiling suite and Pallas tutorials are now the most complete public resource for low-level TPU optimization.

The hub is free to access today at cloud.google.com/products/tpu/tpu-developer. Actual TPU usage is billed through standard Google Cloud pricing. Google has framed this as a living resource that will grow over time, which is worth taking seriously given the pace of new TPU hardware and tooling releases this year.

Trending
  • No trending articles

Comments

avatar

Next Reads