Slow Transition to Low-Dimensional Chaos in Heavy-Tailed Recurrent Neural Networks

TL;DR

This paper investigates how heavy-tailed synaptic weight distributions in recurrent neural networks lead to a slow transition to chaos, revealing a tradeoff between robustness and activity complexity, with implications for biological neural systems.

Contribution

It introduces a theoretical framework for understanding the transition to chaos in finite-size RNNs with heavy-tailed weights, extending mean-field theory to more realistic neural models.

Findings

Heavy-tailed RNNs have a broader parameter regime near the edge of chaos.

The transition from quiescent to chaotic states is slower in heavy-tailed networks.

Heavy tails reduce the Lyapunov dimension, indicating lower activity complexity.

Abstract

Growing evidence suggests that synaptic weights in the brain follow heavy-tailed distributions, yet most theoretical analyses of recurrent neural networks (RNNs) assume Gaussian connectivity. We systematically study the activity of RNNs with random weights drawn from biologically plausible L\'evy alpha-stable distributions. While mean-field theory for the infinite system predicts that the quiescent state is always unstable -- implying ubiquitous chaos -- our finite-size analysis reveals a sharp transition between quiescent and chaotic dynamics. We theoretically predict the gain at which the system transitions from quiescent to chaotic dynamics, and validate it through simulations. Compared to Gaussian networks, heavy-tailed RNNs exhibit a broader parameter regime near the edge of chaos, namely a slow transition to chaos. However, this robustness comes with a tradeoff: heavier tails reduce the Lyapunov dimension of the attractor, indicating lower effective dimensionality. Our results reveal a biologically aligned tradeoff between the robustness of dynamics near the edge of chaos and the richness of high-dimensional neural activity. By analytically characterizing the transition point in finite-size networks -- where mean-field theory breaks down -- we provide a tractable framework for understanding dynamics in realistically sized, heavy-tailed neural circuits.