Dynamical Neural Network: Information and Topology

TL;DR

This paper investigates how the topology of a Hebb neural network affects its mutual information and retrieval capacity, finding optimal configurations depending on connectivity and randomness levels.

Contribution

It introduces an analysis of small-world topologies in Hebb networks to identify conditions for maximal mutual information and optimal retrieval performance.

Findings

Optimal mutual information occurs at near-diluted connectivity for stability.

For dynamic retrieval, the best topology has moderate connectivity when randomness is low.

The study links network topology parameters to information storage and retrieval efficiency.

Abstract

A neural network works as an associative memory device if it has large storage capacity and the quality of the retrieval is good enough. The learning and attractor abilities of the network both can be measured by the mutual information (MI), between patterns and retrieval states. This paper deals with a search for an optimal topology, of a Hebb network, in the sense of the maximal MI. We use small-world topology. The connectivity $\gamma$ ranges from an extremely diluted to the fully connected network; the randomness $\omega$ ranges from purely local to completely random neighbors. It is found that, while stability implies an optimal $MI(\gamma,\omega)$ at $\gamma_{opt}(\omega)\to 0$, for the dynamics, the optimal topology holds at certain $\gamma_{opt}>0$ whenever $0\leq\omega<0.3$.