Dynamical properties of a randomly diluted neural network with variable activity

TL;DR

This paper analyzes the dynamics of a binary neural network with random dilution and variable activity, deriving limits on storage, information, and performance thresholds, including effects of noise and temperature.

Contribution

It provides a comprehensive, exact analysis of the retrieval dynamics in diluted neural networks with variable activity, extending previous results across full parameter ranges.

Findings

Derived restrictions on storage capacity and mutual information during retrieval.

Identified constraints on thresholds for optimal network performance.

Developed a dynamical version of the critical temperature considering noisy updating.

Abstract

The subject of study is a neural network with binary neurons, randomly diluted synapses and variable pattern activity. We look at the system with parallel updating using a probabilistic approach to solve the one step dynamics with one condensed pattern. We derive restrictions on the storage capacity and the mutual information content occuring during the retrieval process. Special focus is on the constraints on the threshold for optimal performance. We also look at the effect of noisy updating, giving a dynamical version of the critical temperature, the corresponding threshold and an approximation for the time evolution for small temperatures. The description is applicable to the whole retrieval process in the limit of strong dilution. The analysis is carried out as exactly as possible and over the full parameter ranges, generalizing some former results.