On the number of limit cycles in diluted neural networks

TL;DR

This paper investigates the number of limit cycles in sparse neural networks, revealing non-monotonic behavior of cycle counts with connectivity, and relates findings to biological memory systems like the hippocampus.

Contribution

It introduces a combined numerical and belief propagation approach to analyze limit cycles in sparse neural networks, highlighting their complex dependence on connectivity.

Findings

Number of cycles varies non-monotonically with connectivity

Sparse networks have more attractors than dense ones

Results relate to biological memory capacity

Abstract

We consider the storage properties of temporal patterns, i.e. cycles of finite lengths, in neural networks represented by (generally asymmetric) spin glasses defined on random graphs. Inspired by the observation that dynamics on sparse systems have more basins of attractions than the dynamics of densely connected ones, we consider the attractors of a greedy dynamics in sparse topologies, considered as proxy for the stored memories. We enumerate them using numerical simulation and extend the analysis to large systems sizes using belief propagation. We find that the logarithm of the number of such cycles is a non monotonic function of the mean connectivity and we discuss the similarities with biological neural networks describing the memory capacity of the hippocampus.