Dense Associative Memory with biased patterns: a Replica Symmetric analysis

TL;DR

This paper analyzes dense higher-order associative memories with biased patterns, revealing how bias affects storage capacity and phase behavior through a replica symmetric analysis.

Contribution

It introduces a modified Hamiltonian and derives self-consistent equations, extending understanding of biased pattern storage in associative memory models.

Findings

Bias reduces storage capacity by a factor of (1-b^2)^P.

Superlinear scaling of capacity with system size is preserved.

Analytical results confirm heuristic predictions about bias effects.

Abstract

We investigate dense higher-order associative memories in the high storage regime when the stored patterns are biased, namely when the entries of the patterns are not symmetrically distributed around zero. In this setting, the standard Hebbian prescription must be modified by recentering and rescaling the pattern entries, and an additional term must be introduced in the Hamiltonian to enforce consistency between the average activity of the network and that of the stored patterns. As a first step, we perform a signal-to-noise analysis in the zero-temperature limit and show that the bias reduces the effective storage capacity through a multiplicative correction factor (1-b^2)^P, while preserving the superlinear scaling with the system size. We then derive the quenched statistical pressure within the Replica Symmetric framework by means of Guerra's interpolation method and obtain the corresponding self consistency equations for the relevant order parameters. The analytical treatment confirms the heuristic prediction of the signal-to-noise argument, showing that the same bias dependent renormalization naturally emerges in the variance of the cross-talk noise. Finally, we discuss the resulting phase behavior of the model and its implications for retrieval performance in the model.