Stability and Geometry of Attractors in Neural Cellular Automata

TL;DR

This paper investigates the attractor dynamics of Neural Cellular Automata (NCA), revealing oscillatory behaviors, early emergence of complex dynamics, and the effects of perturbations, using dynamical systems analysis tools.

Contribution

It provides the first visualizations and spectral analyses of NCA attractors, challenging the assumption that they only learn fixed point attractors.

Findings

NCA exhibits oscillatory, periodic, and quasi-periodic behaviors.

Complex attractor dynamics emerge early during training.

Large perturbations can shift NCA into a different attractor mode.

Abstract

Throughout the literature on Neural Cellular Automata (NCAs), it is often taken for granted that the systems learn attractors. This is shown through evolving the system for many timesteps and noting visual similarity to the goal state. There remain many questions after such an analysis. Namely, what kind of attractors do we have? Is their behavior ordered or chaotic? Can we estimate stability over very long time horizons? What really happens in the attractor when perturbations are applied? In this paper, we present a case study to help answer these questions, with methods drawn from the literature on dynamical systems theory. We use the growing gecko NCA of Mordvintsev et al. (2020) with deterministic cell updates as a case study. To the best of the authors' knowledge, we present the first visualizations of NCA attractor dynamics. We also analyze them using the Lyapunov and Fourier spectra, to reveal that the NCA displays oscillatory, periodic and quasi-periodic behavior, and that these behaviors arise early during training. This challenges the belief that NCAs learn fixed point attractors. Finally, we show that large perturbations to the attractor states can throw the NCAs into a secondary mode separate from the original attractor. We hope that this initial foray into NCA attractor dynamics expands the toolkit for NCA researchers to analyze the robustness and stability of their systems.