Generalized Guerra's interpolation schemes for dense associative neural networks

TL;DR

This paper introduces a novel analytical approach using PDEs to study dense associative neural networks, extending to models with p-body interactions and analyzing phase diagrams and criticality.

Contribution

It develops a new analytical-mechanical method for high-storage neural networks, generalizing Guerra's interpolation to models with complex interactions.

Findings

Explicit free energy expressions for Hopfield and relativistic models

Phase diagrams mapped under replica symmetry

Criticality persists in non-pairwise models

Abstract

In this work we develop analytical techniques to investigate a broad class of associative neural networks set in the high-storage regime. These techniques translate the original statistical-mechanical problem into an analytical-mechanical one which implies solving a set of partial differential equations, rather than tackling the canonical probabilistic route. We test the method on the classical Hopfield model - where the cost function includes only two-body interactions (i.e., quadratic terms) - and on the "relativistic" Hopfield model - where the (expansion of the) cost function includes p-body (i.e., of degree p) contributions. Under the replica symmetric assumption, we paint the phase diagrams of these models by obtaining the explicit expression of their free energy as a function of the model parameters (i.e., noise level and memory storage). Further, since for non-pairwise models ergodicity breaking is non necessarily a critical phenomenon, we develop a fluctuation analysis and find that criticality is preserved in the relativistic model.