Dynamical mean field approach to associative memory model with non-monotonic transfer functions

TL;DR

This paper develops a dynamical mean-field theoretical framework to analyze the retrieval dynamics of a Hopfield associative memory model with non-monotonic transfer functions, providing quantitative insights into its performance and states.

Contribution

It introduces a novel dynamical mean-field approach to analyze non-monotonic transfer functions in associative memory models, which was previously only qualitatively understood.

Findings

Accurately characterizes macroscopic dynamical properties of the non-monotonic model

Derives conditions for retrieval states in the non-monotonic setting

Clarifies the relationship between retrieval states and previous models

Abstract

The Hopfield associative memory model stores random patterns in synaptic couplings according to Hebb's rule and retrieves them through gradient descent on an energy function. This conventional setting, where neurons are assumed to have monotonic transfer functions, has been central to understanding associative memory. Morita (1993, Neural Netw. 6 115), however, showed that introducing non-monotonic transfer functions can dramatically enhance retrieval performance. While this phenomenon has been qualitatively examined, a full quantitative theory remains elusive due to the difficulty of analysis in the absence of an underlying energy function. In this work, we apply dynamical mean-field theory to the discrete-time synchronous retrieval dynamics of the non-monotonic model, which succeeds in accurately characterizing its macroscopic dynamical properties. We also derive conditions for retrieval states, and clarify their relation to previous studies. Our results provide new insights into the non-equilibrium retrieval dynamics of associative memory models.