Learning Continuous Chaotic Attractors with a Reservoir Computer

TL;DR

This paper demonstrates how a reservoir computer can learn and abstract a continuum of dynamical attractors, including chaotic ones, through a new theoretical framework combining synchronization and feedback dynamics.

Contribution

It introduces a novel method for training RNNs to abstract continuous attractor memories and provides a theoretical explanation for this process.

Findings

Reservoir computer successfully learns a continuum of attractors.

The learned attractors include stable limit cycles and chaotic Lorenz attractors.

A new theory explains the abstraction mechanism via synchronization and feedback.

Abstract

Neural systems are well known for their ability to learn and store information as memories. Even more impressive is their ability to abstract these memories to create complex internal representations, enabling advanced functions such as the spatial manipulation of mental representations. While recurrent neural networks (RNNs) are capable of representing complex information, the exact mechanisms of how dynamical neural systems perform abstraction are still not well-understood, thereby hindering the development of more advanced functions. Here, we train a 1000-neuron RNN -- a reservoir computer (RC) -- to abstract a continuous dynamical attractor memory from isolated examples of dynamical attractor memories. Further, we explain the abstraction mechanism with new theory. By training the RC on isolated and shifted examples of either stable limit cycles or chaotic Lorenz attractors, the RC learns a continuum of attractors, as quantified by an extra Lyapunov exponent equal to zero. We propose a theoretical mechanism of this abstraction by combining ideas from differentiable generalized synchronization and feedback dynamics. Our results quantify abstraction in simple neural systems, enabling us to design artificial RNNs for abstraction, and leading us towards a neural basis of abstraction.