Thresholds in layered neural networks with variable activity

TL;DR

This paper investigates how different threshold strategies affect the retrieval quality of layered neural networks with variable activity, especially at different temperatures, using numerical analysis focused on low activity networks.

Contribution

It compares self-controlled and externally optimized thresholds in layered neural networks, highlighting their impact on retrieval performance at various temperatures.

Findings

Self-controlled thresholds improve retrieval stability.

Optimized thresholds maximize mutual information.

Low activity networks benefit from threshold tuning.

Abstract

The inclusion of a threshold in the dynamics of layered neural networks with variable activity is studied at arbitrary temperature. In particular, the effects on the retrieval quality of a self-controlled threshold obtained by forcing the neural activity to stay equal to the activity of the stored paterns during the whole retrieval process, are compared with those of a threshold chosen externally for every loading and every temperature through optimisation of the mutual information content of the network. Numerical results, mostly concerning low activity networks are discussed.