Retrieval dynamics of neural networks for sparsely coded sequential patterns

TL;DR

This paper investigates the dynamic retrieval process of neural networks storing sparsely coded sequential patterns, providing theoretical predictions for capacity and robustness, and highlighting the influence of activity control methods.

Contribution

It introduces a statistical neurodynamics approach to analyze the retrieval dynamics of sparsely coded sequential patterns, extending understanding beyond static pattern storage.

Findings

The theory accurately predicts storage capacity and basin of attraction.

The basin of attraction depends on activity control methods.

Robustness decreases slightly with increased sparsity.

Abstract

It is well known that a sparsely coded network in which the activity level is extremely low has intriguing equilibrium properties. In the present work, we study the dynamical properties of a neural network designed to store sparsely coded sequential patterns rather than static ones. Applying the theory of statistical neurodynamics, we derive the dynamical equations governing the retrieval process which are described by some macroscopic order parameters such as the overlap. It is found that our theory provides good predictions for the storage capacity and the basin of attraction obtained through numerical simulations. The results indicate that the nature of the basin of attraction depends on the methods of activity control employed. Furthermore, it is found that robustness against random synaptic dilution slightly deteriorates with the degree of sparseness.