AI
How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text
Publication: Hugging Face Daily Papers
Author: Not available in collected metadata
Source published: 29/09/2026, 20:00:00 UTC
Briefing created: 01/10/2026, 23:16:10 UTC
Based on a feed excerpt; full article content has not been acquired. Source evidence retrieved 01/10/2026, 14:18:24 UTC.
Briefings
Research published in September 2026 reveals that AI-generated web text significantly impacts language model pretraining. Pangram's FineWeb filtering identified 27.5% of June 2026 tokens as AI-generated, rising to 31.1% by August. Experiments with 800 language models show that AI tokens initially lower loss on human text but cause harm once saturation is reached, contradicting established scaling laws. A new scaling law was proposed to handle this dynamic value, predicting up to 41% lower error on smaller models compared to existing laws.
Key points
- Pangram's FineWeb filtering identified 27.5% of June 2026 web tokens as AI-generated, rising to 31.1% by August 2026.
- Adding AI tokens to pretraining data initially lowers loss on human text but causes harm once saturation is reached.
Context and limitations
Scaling law predictions fail for data-starved models; specific loss reversal thresholds vary by model size and budget; validation loss on AI text remains unmeasured.
Generated by Bittrees-hosted qwen35-2b sha256:aaf42c8b7c3c on 01/10/2026, 23:16:10 UTC. Based on publisher-provided feed evidence; may not cover the full original article. Check the source for details.
View archived briefing on IPFS
bafybeialbeg5uguiuprxe3jo3ey5ktm4hewu2yy5dtg5zwxjx4czcuv5pe