Mean Field Theory of Dynamical Systems Driven by External Signals

TL;DR

This paper develops a mean field theory for echo state networks, modeling their dynamics under external signals, and compares theoretical predictions with numerical simulations to understand their behavior.

Contribution

It introduces a mean field framework for echo state networks driven by external signals, capturing their collective dynamics and stability properties.

Findings

The theory predicts the network's steady state under multiple signals.

It characterizes nonstationary dynamics driven by a single external signal.

The largest Lyapunov exponent matches numerical results.

Abstract

Dynamical systems driven by strong external signals are ubiquituous in nature and engineering. Here we study "echo state networks", networks of a large number of randomly connected nodes, which represent a simple model of a neural network, and have important applications in machine learning. We develop a mean field theory of echo state networks. The dynamics of the network is captured by the evolution law, similar to a logistic map, for a single collective variable. When the network is driven by many independent external signals, this collective variable reaches a steady state. But when the network is driven by a single external signal, the collective variable is nonstationnary but can be characterised by its time averaged distribution. The predictions of the mean field theory, including the value of the largest Lyaponuov exponent, are compared with the numerical integration of the equations of motion.