---
title: "Pangram Labs Warns AI Web Text Could Triple Pretraining Costs by 2028"
subtitle: "Pangram Labs trained 800 language models to measure how AI-generated web text warps pretraining, and built a new scaling law to price each token."
url: "https://alphasignal.ai/news/pangram-labs-warns-ai-web-text-could-triple-pretraining-costs-by-2028"
author: "AlphaSignal Newsroom"
author_url: "https://alphasignal.ai/editorial-team"
company: "AlphaSignal"
company_url: "https://alphasignal.ai/company/alphasignal"
published: "2026-09-30T17:50:22.000Z"
updated: "2026-10-02T12:05:27.203Z"
type: "Paper"
topics:
  - "Training Infra"
  - "Data"
subtopics:
  - "Scaling Laws"
  - "Pretraining"
  - "Synthetic Data"
upvotes: 93
source_url: "https://arxiv.org/abs/2609.40295"
comments: 0
truncated: true
---

# Pangram Labs Warns AI Web Text Could Triple Pretraining Costs by 2028

> Pangram Labs trained 800 language models to measure how AI-generated web text warps pretraining, and built a new scaling law to price each token.

# AI Web Text Changes the Economics of Pretraining

In an August 2026 web crawl, researchers from Pangram Labs and the University of Maryland classified 31% of tokens as AI-generated. A [new study](https://arxiv.org/abs/2609.40295) then trained 800 language models on controlled mixtures of human and AI web text to measure how each source affects next-token prediction loss. The result depends on the training regime: generated tokens can help data-starved models, but their marginal value saturates and eventually becomes negative as human-data budgets and model sizes grow. Training plans based on the [Chinchilla scaling law](https://arxiv.org/abs/2203.15556) miss that reversal because they assign equal value to every token.

The team’s estimated AI share rose from roughly 10% in June 2024 to 27.5% in June 2026 and 31% two months later. Common quality filters increased that concentration. FineWeb’s pipeline retained AI-labeled documents at 2.3 times the rate of human-labeled documents, while DCLM’s retained them at 9.8 times the rate. A filtered crawl can therefore contain a higher proportion of generated text than its source crawl.

![AI share of web tokens over time, the marginal value of AI tokens, and estimated compute penalties](https://imagedelivery.net/kokADdxuw1FCXyR2p8D2CQ/af7a7a92-0ba1-445d-f73f-3f36618f9300/public) 

The study connects the rising share of AI web text with changes in token value and pretraining compute.

## Equal-token math reaches its limit

Earlier synthetic-data research largely examined purpose-built training corpora or recursive model-collapse experiments in which models learn from their own outputs. Wild web text has a broader origin. It comes from systems such as GPT-4o, Claude, and Llama, may include human edits, and appears alongside human writing across many topics and formats. The WildAI experiments isolate this material in controlled training mixtures.

_Preview only. Full text requires an AlphaSignal Pro account._

## Sources

- [How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text](https://arxiv.org/html/2609.40295)
- [pangramlabs/WildAI on GitHub](https://github.com/pangramlabs/WildAI)
- [The AI Data Satiation Point: Why Wild AI Text is Breaking Chinchilla Scaling Laws](https://dev.to/prabhakar_chaudhary_7afe4/the-ai-data-satiation-point-why-wild-ai-text-is-breaking-chinchilla-scaling-laws-3e2d)
- [Jenna Russell on X](https://twitter.com/jennajrussell/status/2105679818209796544)