About the de Almeida-Thouless line in neural networks

TL;DR

This paper introduces a simple, rigorous method to detect the onset of replica-symmetry breaking in neural networks and disordered systems, avoiding complex full replica analysis and applicable to systems with discontinuous phase transitions.

Contribution

The authors develop a new method based on free-energy expansion around the RS solution, enabling detection of RS instability without full replica calculations, applicable to various disordered models.

Findings

Successfully applied to Hopfield and neural networks with Hebbian interactions.

Recovered known AT lines in SK and Ising P-spin models as special cases.

Extended to systems with discontinuous phase transitions, such as the spherical P-spin model.

Abstract

In this work we present a rigorous and straightforward method to detect the onset of the instability of replica-symmetric theories in information processing systems, which does not require a full replica analysis as in the method originally proposed by de Almeida and Thouless for spin glasses. The method is based on an expansion of the free-energy obtained within one-step of replica symmetry breaking (RSB) around the RS value. As such, it requires solely continuity and differentiability of the free-energy and it is robust to be applied broadly to systems with quenched disorder. We apply the method to the Hopfield model and to neural networks with multi-node Hebbian interactions, as case studies. In the appendices we test the method on the Sherrington-Kirkpatrick and the Ising P-spin models, recovering the AT lines known in the literature for these models, as a special limit, which corresponds to assuming that the transition from the RS to the RSB phase can be obtained by varying continuously the order parameters. Our method provides a generalization of the AT approach, which does not rely on this limit and can be applied to systems with discontinuous phase transitions, as we show explicitly for the spherical P-spin model, recovering the known RS instability line.