Constraints for the order parameters in analogical neural networks

TL;DR

This paper investigates the equilibrium properties of a Gaussian-pattern Hopfield neural network, revealing constraints on order parameters similar to those in spin glasses, through advanced statistical mechanics techniques.

Contribution

It introduces new Ghirlanda-Guerra-like identities for neural network overlaps, extending constraints known in spin glass theory to neural models.

Findings

Existence of constraints on order parameters near phase transitions

Derivation of Ghirlanda-Guerra-like identities for neural overlaps

Identification of new identities involving noise explicitly

Abstract

In this paper we study, via equilibrium statistical mechanics, the properties of the internal energy of an Hopfield neural network whose patterns are stored continuously (Gaussian distributed). The model is shown to be equivalent to a bipartite spin glass in which one party is given by dichotomic neurons and the other party by Gaussian spin variables. Dealing with replicated systems, beyond the Mattis magnetization, we introduce two overlaps, one for each party, as order parameters of the theory: The first is a standard overlap among neural configurations on different replicas, the second is an overlap among the Gaussian spins of different replicas. The aim of this work is to show the existence of constraints for these order parameters close to ones found in many other complex systems as spin glasses and diluted networks: we find a class of Ghirlanda-Guerra-like identities for both the overlaps, generalizing the well known constraints to the neural networks, as well as new identities where noise is involved explicitly.