Adaptive thresholds for neural networks with synaptic noise

TL;DR

This paper investigates adaptive thresholds in neural networks with synaptic noise, demonstrating that self-adjusting thresholds enhance retrieval quality, storage capacity, and robustness across different architectures.

Contribution

It introduces a method for adaptive thresholds that automatically adjust during recall, improving neural network performance with synaptic noise.

Findings

Enhanced storage capacity and basins of attraction.

Improved mutual information content.

Robust autonomous network functioning.

Abstract

The inclusion of a macroscopic adaptive threshold is studied for the retrieval dynamics of both layered feedforward and fully connected neural network models with synaptic noise. These two types of architectures require a different method to be solved numerically. In both cases it is shown that, if the threshold is chosen appropriately as a function of the cross-talk noise and of the activity of the stored patterns, adapting itself automatically in the course of the recall process, an autonomous functioning of the network is guaranteed. This self-control mechanism considerably improves the quality of retrieval, in particular the storage capacity, the basins of attraction and the mutual information content.