A layered neural network with three-state neurons optimizing the mutual information

TL;DR

This paper analyzes a layered neural network with three-state neurons that optimizes mutual information, revealing phase diagrams and improved capacity over previous models, with insights into network dynamics and phase behavior.

Contribution

It introduces an exactly solvable layered neural network model with three-state neurons that optimizes mutual information, showing enhanced capacity and detailed phase analysis.

Findings

The model exhibits pattern retrieval, fluctuation retrieval, and spin-glass phases.

It achieves larger critical capacity and information content than previous three-state models.

Flow dynamics are affected by saddle-point solutions, slowing convergence.

Abstract

The time evolution of an exactly solvable layered feedforward neural network with three-state neurons and optimizing the mutual information is studied for arbitrary synaptic noise (temperature). Detailed stationary temperature-capacity and capacity-activity phase diagrams are obtained. The model exhibits pattern retrieval, pattern-fluctuation retrieval and spin-glass phases. It is found that there is an improved performance in the form of both a larger critical capacity and information content compared with three-state Ising-type layered network models. Flow diagrams reveal that saddle-point solutions associated with fluctuation overlaps slow down considerably the flow of the network states towards the stable fixed-points.