Attractor metadynamics in terms of target points in slow-fast systems: adiabatic vs. symmetry protected flow in a recurrent neural network

TL;DR

This paper introduces the concept of target points as a practical subset of the critical manifold in high-dimensional slow-fast dynamical systems, enabling analysis of neural network dynamics and distinguishing adiabatic regimes from non-adiabatic ones.

Contribution

It proposes using target points as a computationally accessible tool to analyze the critical manifold in large neural networks with slow-fast dynamics.

Findings

Target points effectively characterize the system's slow-fast behavior.

Scaling of Euclidean distance reveals adiabatic versus non-adiabatic regimes.

Framework aids in understanding state switching in neural networks.

Abstract

In dynamical systems with distinct time scales the time evolution in phase space may be influenced strongly by the fixed points of the fast subsystem. Orbits then typically follow these points, performing in addition rapid transitions between distinct branches on the time scale of the fast variables. As the branches guide the dynamics of a system along the manifold of former fixed points, they are considered transiently attracting states and the intermittent transitions between branchescorrespond to state switching within transient-state dynamics. A full characterization of the set of former fixed points, the critical manifold, tends to be difficult in high-dimensional dynamical systems such as large neural networks. Here we point out that an easily computable subset of the critical manifold, the set of target points, can be used as a reference for the investigation of high-dimensional slow-fast systems. The set of target points corresponds in this context to the adiabatic projection of a given orbit to the critical manifold. Applying our framework to a simple recurrent neural network, we find that the scaling relation of the Euclidean distance between the trajectory and its target points with the control parameter of the slow time scale allows to distinguish an adiabatic regime from a state that is effectively independent from target points.