Three-state neural network: from mutual information to the hamiltonian

TL;DR

This paper derives an exact expression for the mutual information in a three-state neural network, leading to a Hamiltonian similar to the Blume-Emery-Griffiths model, which enhances retrieval capacity and information retention.

Contribution

It introduces a Hamiltonian based on mutual information for three-state neural networks, extending understanding of their retrieval properties and dynamics.

Findings

Information can persist without pattern overlap.

The retrieval capacity exceeds that of Hopfield networks.

Phase diagrams of the diluted network are provided.

Abstract

The mutual information, I, of the three-state neural network can be obtained exactly for the mean-field architecture, as a function of three macroscopic parameters: the overlap, the neural activity and the {\em activity-overlap}, i.e. the overlap restricted to the active neurons. We perform an expansion of I on the overlap and the activity-overlap, around their values for neurons almost independent on the patterns. From this expansion we obtain an expression for a Hamiltonian which optimizes the retrieval properties of this system. This Hamiltonian has the form of a disordered Blume-Emery-Griffiths model. The dynamics corresponding to this Hamiltonian is found. As a special characteristic of such network, we see that information can survive even if no overlap is present. Hence the basin of attraction of the patterns and the retrieval capacity is much larger than for the Hopfield network. The extreme diluted version is analized, the curves of information are plotted and the phase diagrams are built.