AutomataGPT: Forecasting and Ruleset Inference for Two-Dimensional Cellular Automata

TL;DR

AutomataGPT is a transformer-based model trained on simulated cellular automata data that achieves high accuracy in forecasting and inferring rules, enabling data-driven modeling of complex dynamical systems across various scientific domains.

Contribution

The paper introduces AutomataGPT, a transformer model capable of accurately forecasting and inferring rules of two-dimensional cellular automata without handcrafted priors, demonstrating strong generalization.

Findings

Achieves 98.5% one-step forecast accuracy on unseen CA rules.

Reconstructs governing rules with up to 96% functional accuracy.

Demonstrates potential for data-efficient modeling of real-world phenomena.

Abstract

Cellular automata (CA) provide a minimal formalism for investigating how simple local interactions generate rich spatiotemporal behavior in domains as diverse as traffic flow, ecology, tissue morphogenesis and crystal growth. However, automatically discovering the local update rules for a given phenomenon and using them for quantitative prediction remains challenging. Here we present AutomataGPT, a decoder-only transformer pretrained on around 1 million simulated trajectories that span 100 distinct two-dimensional binary deterministic CA rules on toroidal grids. When evaluated on previously unseen rules drawn from the same CA family, AutomataGPT attains 98.5% perfect one-step forecasts and reconstructs the governing update rule with up to 96% functional (application) accuracy and 82% exact rule-matrix match. These results demonstrate that large-scale pretraining over wider regions of rule space yields substantial generalization in both the forward (state forecasting) and inverse (rule inference) problems, without hand-crafted priors. By showing that transformer models can faithfully infer and execute CA dynamics from data alone, our work lays the groundwork for abstracting real-world dynamical phenomena into data-efficient CA surrogates, opening avenues in biology, tissue engineering, physics and AI-driven scientific discovery.