Gardner formula for Ising perceptron models at small densities

TL;DR

This paper provides a new proof for the convergence of the free energy in the Ising perceptron model at small densities, extending previous results to more general activation functions using probabilistic methods.

Contribution

It introduces a novel proof technique for the perceptron free energy convergence, applicable to a broader class of activation functions beyond previous restrictions.

Findings

Convergence of free energy to the replica symmetric formula at small densities.

Extension of results to all bounded above activation functions with variance bounds.

New concentration results for the perceptron model with unbounded below activation functions.

Abstract

We consider the Ising perceptron model with N spins and M = N*alpha patterns, with a general activation function U that is bounded above. For U bounded away from zero, or U a one-sided threshold function, it was shown by Talagrand (2000, 2011) that for small densities alpha, the free energy of the model converges in the large-N limit to the replica symmetric formula conjectured in the physics literature (Krauth--Mezard 1989, see also Gardner--Derrida 1988). We give a new proof of this result, which covers the more general class of all functions U that are bounded above and satisfy a certain variance bound. The proof uses the (first and second) moment method conditional on the approximate message passing iterates of the model. In order to deduce our main theorem, we also prove a new concentration result for the perceptron model in the case where U is not bounded away from zero.