Neural Networks as Geometric Chaotic Maps

TL;DR

This paper demonstrates that neural networks can effectively model chaotic systems by becoming structurally chaotic, using a geometric perspective to explain their ability to reconstruct and predict complex chaotic behaviors.

Contribution

It introduces a geometric framework to understand how neural networks learn chaos, showing they perform geometric operations indicative of chaos, which explains their effectiveness in modeling chaotic dynamics.

Findings

Neural networks can reconstruct strange attractors from limited data.

Trained networks can extrapolate beyond training data boundaries.

Networks exhibit geometric operations like stretching and rotation indicative of chaos.

Abstract

The use of artificial neural networks as models of chaotic dynamics has been rapidly expanding. Still, a theoretical understanding of how neural networks learn chaos is lacking. Here, we employ a geometric perspective to show that neural networks can efficiently model chaotic dynamics by becoming structurally chaotic themselves. We first confirm neural network's efficiency in emulating chaos by showing that a parsimonious neural network trained only on few data points can reconstruct strange attractors, extrapolate outside training data boundaries, and accurately predict local divergence rates. We then posit that the trained network's map comprises sequential geometric stretching, rotation, and compression operations. These geometric operations indicate topological mixing and chaos, explaining why neural networks are naturally suitable to emulate chaotic dynamics.