--- title: "Epoch AI Warns Token Demand Is Outpacing Global GPU Supply by 10x" subtitle: "Epoch AI models every Blackwell GPU on Earth and finds token demand growing 10x/year while supply grows just 3-4x — a crunch may already be here" url: "https://alphasignal.ai/news/epoch-ai-warns-token-demand-is-outpacing-global-gpu-supply-by-10x" author: "AlphaSignal Newsroom" author_url: "https://alphasignal.ai/editorial-team" company: "Epoch AI" company_url: "https://alphasignal.ai/company/epochairesearch" published: "2026-05-26T20:35:47.000Z" updated: "2026-06-08T19:33:17.175Z" type: "News" topics: - "Training Infra" - "Llms" subtopics: - "Scaling Laws" - "Mixture Of Experts" upvotes: 480 read_time: "4 min" source_url: "https://epoch.ai/gradient-updates/is-a-compute-crunch-coming" comments: 0 --- # Epoch AI Warns Token Demand Is Outpacing Global GPU Supply by 10x > Epoch AI models every Blackwell GPU on Earth and finds token demand growing 10x/year while supply grows just 3-4x — a crunch may already be here The AI industry has spent the last two years in a frenzy of hardware investment, but a new analysis from [Epoch AI](https://epoch.ai/gradient-updates/is-a-compute-crunch-coming) asks the uncomfortable question: is all that silicon actually enough? Researchers Luke Emberson and Jaime Sevilla built a bottom-up model of global inference capacity and stacked it against the best available proxies for token demand. The verdict is sobering , demand appears to be outrunning supply by a wide margin, and the gap is widening. ## The trillion-dollar question Hyperscalers have been racing to build massive data centers, spending hundreds of billions in the process. The St. Louis Fed estimates that AI-related investment contributed about 1 percentage point , nearly 40% of the total , to US real GDP growth in the first three quarters of 2025, exceeding the IT investment contribution at the height of the dot-com boom. Whether the current AI buildout constitutes a bubble depends largely on whether there will be sufficient demand for the computing infrastructure being built. Epoch AI estimates supply is growing 3 to 4 times per year, while demand for tokens is growing roughly 10 times per year , suggesting a compute crunch is nearing, if not already here. ## How they counted every token the world could serve The analysis is grounded in a specific, concrete setup: serving [Kimi K2.6](https://build.nvidia.com/moonshotai/kimi-k2.6/modelcard) , a 1-trillion-parameter mixture-of-experts model with 32 billion active parameters , on all of the world's Nvidia GB200 and GB300 chips. That means 1.9 million and 1.5 million individual GPUs respectively, representing roughly 40% of aggregate global compute on a FLOP/s basis. Three request profiles were modeled: - **General usage:** 8,000 input tokens, 1,000 output tokens - **Agentic (medium):** 25,000 input tokens, 1,000 output tokens - **Agentic (long):** 128,000 input tokens, 1,000 output tokens The researchers built a theoretical model of inference from first principles, then calibrated it against real-world data from SemiAnalysis's [InferenceX dashboard](https://inferencex.semianalysis.com/) , a repository of thousands of real inference experiments. Calibration revealed 65% compute efficiency and 30% bandwidth efficiency in practice, significantly below peak specs. ![Scatter plot comparing theoretical vs calibrated throughput predictions across GPU types](https://epoch.ai/assets/images/gradient-updates/2026/is-a-compute-crunch-coming/uncalibrated-vs-calibrated-throughput-predictions.png) ## Prefill, decode, and the two tricks that matter The model breaks inference into its two core phases. During **prefill**, all input tokens are processed in parallel to populate the KV cache (a memory structure that stores intermediate attention computations so they don't need to be recomputed). During **decode**, output tokens are generated one at a time. These two phases have very different bottlenecks: prefill is compute-bound (the GPU math units are the limiting factor), while decode is bandwidth-bound (the bottleneck is how fast weights and the KV cache can be loaded from memory). ![Diagram comparing naive inference versus chunked prefill approaches](https://epoch.ai/assets/images/gradient-updates/2026/is-a-compute-crunch-coming/naive-vs-chunked-prefill.png) Two optimization techniques are layered on top of this baseline: - **Chunked prefill:** Because prefill is compute-bound and decode is bandwidth-bound, the two phases leave complementary resources idle. Chunked prefill interleaves them , while one batch is decoding (using bandwidth), the next batch's prefill computations run on the spare compute units. The result is that total cycle time collapses to the maximum of total compute work or total bandwidth work, not the sum. - **Speculative decoding:** A small "draft" model proposes several tokens ahead, and the main model verifies them all in a single forward pass. When the draft is right, you get multiple output tokens per pass. Using state-of-the-art implementations, this boosts decoding throughput at a fixed batch size by roughly 3x, with a combined effect on total throughput of 1.6 to 2x. ## The numbers: how much can the world actually serve? The final estimate suggests the world's Blackwell GPUs could currently deliver a combined 500 million to 20 billion output tokens per second, or between 150,000 and 7 million tokens per month for each person on Earth. The wide range is driven almost entirely by context length , longer inputs crush throughput because attention costs scale quadratically with sequence length. | Request type (input:output) | Output tokens per second (global) | | --- | --- | | 8k:1k (general) | ~20 billion | | 25k:1k (agentic, medium) | ~6 billion | | 128k:1k (agentic, long) | ~500 million | Google, likely the most avid token producer today through Search summaries, recently claimed it was serving 1.2 billion tokens per second across all its platforms. Assuming an 8 ## Sources - [X post by @epochairesearch](https://x.com/epochairesearch/status/2059372951338909717) - [Is a compute crunch coming? | Epoch AI](https://epoch.ai/gradient-updates/is-a-compute-crunch-coming) - [The Epoch Brief - June 1, 2026 | Epoch AI Substack](https://epochai.substack.com/p/the-epoch-brief-june-1-2026) - [Is a compute crunch coming? | Epoch AI Substack](https://epochai.substack.com/p/is-a-compute-crunch-coming)