Thermal Robustness of Retrieval in Dense Associative Memories: LSE vs LSR Kernels

TL;DR

This paper investigates the thermal robustness of retrieval in dense associative memories using Monte Carlo simulations, comparing two kernels and revealing distinct temperature-dependent retrieval behaviors.

Contribution

It introduces a detailed comparison of LSE and LSR kernels in dense associative memories, highlighting their different thermal robustness properties.

Findings

LSE kernel maintains retrieval at high temperatures at low load.

LSR kernel exhibits a finite load threshold for perfect retrieval at any temperature.

Theoretical predictions align with simulation results within the retrieval basin.

Abstract

Understanding whether retrieval in dense associative memories survives thermal noise is essential for bridging zero-temperature capacity proofs with the finite-temperature conditions of practical inference and biological computation. We use Monte Carlo simulations to map the retrieval phase boundary of two continuous dense associative memories (DAMs) on the $N$-sphere with an exponential number of stored patterns $M = e^{\alpha N}$: a log-sum-exp (LSE) kernel and a log-sum-ReLU (LSR) kernel. Both kernels share the zero-temperature critical load $\alpha_c(0)=0.5$, but their finite-temperature behavior differs markedly. The LSE kernel sustains retrieval at arbitrarily high temperatures for sufficiently low load, whereas the LSR kernel exhibits a finite support threshold below which retrieval is perfect at any temperature; for typical sharpness values this threshold approaches $\alpha_c$, making retrieval nearly perfect across the entire load range. We also compare the measured equilibrium alignment with analytical Boltzmann predictions within the retrieval basin.