A Step Towards Uncovering The Structure of Multistable Neural Networks

TL;DR

This paper analyzes how the structure of multistable neural networks influences their stable states, providing exact analytical results on network topology, stability, and pattern dynamics using a simplified Heaviside activation model.

Contribution

It introduces a novel analytical framework for understanding multistability in recurrent neural networks with non-smooth activation functions, linking network structure to stability and pattern behavior.

Findings

Derived exact conditions for stable equilibria based on network topology.

Established relationships between weight matrix properties and pattern stability.

Provided insights into pattern completion and coupling mechanisms.

Abstract

We study how the connectivity within a recurrent neural network determines and is determined by the multistable solutions of network activity. To gain analytic tractability we let neural activation be a non-smooth Heaviside step function. This nonlinearity partitions the phase space into regions with different, yet linear dynamics. In each region either a stable equilibrium state exists, or network activity flows to outside of the region. The stable states are identified by their semipositivity constraints on the synaptic weight matrix. The restrictions can be separated by their effects on the signs or the strengths of the connections. Exact results on network topology, sign stability, weight matrix factorization, pattern completion and pattern coupling are derived and proven. Our work may lay the foundation for multistability in more complex recurrent neural networks.