Transformer Learning of Chaotic Collective Dynamics in Many-Body Systems

TL;DR

This paper demonstrates that self-attention-based transformers can effectively model chaotic collective dynamics in many-body systems from time-series data, capturing statistical properties despite long-term prediction divergence.

Contribution

It introduces a transformer framework that learns non-Markovian reduced descriptions of chaotic dynamics, outperforming traditional recurrent models in capturing statistical features.

Findings

Transformer reproduces statistical climate of chaos

Reweights long-range temporal correlations effectively

Outperforms recurrent architectures in modeling chaos

Abstract

Learning reduced descriptions of chaotic many-body dynamics is fundamentally challenging: although microscopic equations are Markovian, collective observables exhibit strong memory and exponential sensitivity to initial conditions and prediction errors. We show that a self-attention-based transformer framework provides an effective approach for modeling such chaotic collective dynamics directly from time-series data. By selectively reweighting long-range temporal correlations, the transformer learns a non-Markovian reduced description that overcomes intrinsic limitations of conventional recurrent architectures. As a concrete demonstration, we study the one-dimensional semiclassical Holstein model, where interaction quenches induce strongly nonlinear and chaotic dynamics of the charge-density-wave order parameter. While pointwise predictions inevitably diverge at long times, the transformer faithfully reproduces the statistical "climate" of the chaos, including temporal correlations and characteristic decay scales. Our results establish self-attention as a powerful mechanism for learning effective reduced dynamics in chaotic many-body systems.