AI
Learning Chaos Without Seeing Chaos: Extrapolation of Global Dynamics in Autoregressive Transformers
Publication: Hugging Face Daily Papers
Author: Not available in collected metadata
Source published: 29/09/2026, 20:00:00 UTC
Briefing created: 02/10/2026, 11:46:36 UTC
Based on a feed excerpt; full article content has not been acquired. Source evidence retrieved 02/10/2026, 02:18:25 UTC.
Briefings
Autoregressive transformers trained on local trajectories of non-linear dynamical systems successfully recover global dynamics like period-doubling cascades and chaos when evaluated at unseen parameters. The logistic map model reproduces period doublings up to 128, matching the Feigenbaum constant to within 5e-4. Causal interventions show attention processes control parameters to shape closed-loop dynamics, suggesting narrow local observation suffices for generalization.
Key points
- Autoregressive transformers trained from scratch on local trajectories of non-linear dynamical systems successfully recover global dynamics like period-doubling cascades and chaos when evaluated at unseen parameters.
- The logistic map model reproduces period doublings up to period 128, yielding a finite-order scaling ratio of 4.6687 that matches the Feigenbaum constant to within 5e-4.
Context and limitations
Results are based on small autoregressive transformers trained from scratch on trajectories sampled from restricted parameter regimes of several non-linear dynamical systems. Evaluation occurs at parameters far outside the training distribution, and causal interventions were limited to investigating parameter processing through attention.
Generated by Bittrees-hosted qwen35-2b sha256:aaf42c8b7c3c on 02/10/2026, 11:46:36 UTC. Based on publisher-provided feed evidence; may not cover the full original article. Check the source for details.
View archived briefing on IPFS
bafybeigml4oaosr3vtfjp63rbr2m4x3f75c4ifbqeyk5ufri5vl34gyhsm