Network Hierarchy and Pattern Recovery in Directed Sparse Hopfield Networks

TL;DR

This paper investigates how the hierarchical structure of directed, sparse Hopfield networks influences their pattern recovery capabilities, revealing that controlling trophic coherence and identifying key nodes enhances performance.

Contribution

It introduces the use of Trophic Analysis to characterize hierarchy in neural networks and demonstrates how network topology affects pattern recovery and control.

Findings

Small subset of neurons can control the system based on trophic levels.

Tuning trophic coherence improves pattern recovery performance.

Hierarchical structure insights relate to biological and artificial networks.

Abstract

Many real-world networks are directed, sparse and hierarchical, with a mixture of feed-forward and feedback connections with respect to the hierarchy. Moreover, a small number of 'master' nodes are often able to drive the whole system. We study the dynamics of pattern presentation and recovery on sparse, directed, Hopfield-like neural networks using Trophic Analysis to characterise their hierarchical structure. This is a recent method which quantifies the local position of each node in a hierarchy (trophic level) as well as the global directionality of the network (trophic coherence). We show that even in a recurrent network, the state of the system can be controlled by a small subset of neurons which can be identified by their low trophic levels. We also find that performance at the pattern recovery task can be significantly improved by tuning the trophic coherence and other topological properties of the network. This may explain the relatively sparse and coherent structures observed in the animal brain, and provide insights for improving the architectures of artificial neural networks. Moreover, we expect that the principles we demonstrate, through numerical analysis, here will be relevant for a broad class of system whose underlying network structure is directed and sparse, such as biological, social or financial networks.