The Relativistic Hopfield network: rigorous results

TL;DR

This paper rigorously analyzes the statistical mechanics of the relativistic Hopfield network, establishing the existence of its free-energy limit and deriving self-consistent equations for overlaps, thus providing a solid theoretical foundation for this generalized neural network model.

Contribution

It provides a rigorous statistical mechanical analysis of the relativistic Hopfield model, including proof of the free-energy limit and explicit overlap equations, extending previous results.

Findings

Existence of the infinite volume free-energy limit.

Explicit expression for the free-energy in terms of overlaps.

Self-consistent equations for the Mattis overlaps.

Abstract

The relativistic Hopfield model constitutes a generalization of the standard Hopfield model that is derived by the formal analogy between the statistical-mechanic framework embedding neural networks and the Lagrangian mechanics describing a fictitious single-particle motion in the space of the tuneable parameters of the network itself. In this analogy the cost-function of the Hopfield model plays as the standard kinetic-energy term and its related Mattis overlap (naturally bounded by one) plays as the velocity. The Hamiltonian of the relativisitc model, once Taylor-expanded, results in a P-spin series with alternate signs: the attractive contributions enhance the information-storage capabilities of the network, while the repulsive contributions allow for an easier unlearning of spurious states, conferring overall more robustness to the system as a whole. Here we do not deepen the information processing skills of this generalized Hopfield network, rather we focus on its statistical mechanical foundation. In particular, relying on Guerra's interpolation techniques, we prove the existence of the infinite volume limit for the model free-energy and we give its explicit expression in terms of the Mattis overlaps. By extremizing the free energy over the latter we get the generalized self-consistent equations for these overlaps, as well as a picture of criticality that is further corroborated by a fluctuation analysis. These findings are in full agreement with the available previous results.