Partial annealing and pattern decorrelation in associative neural networks

TL;DR

This paper investigates how partial annealing in associative neural networks, modeled after Hopfield networks, can reduce pattern interference and improve memory retrieval by adaptively decorrelating stored patterns.

Contribution

It introduces a novel partial annealing framework with a tunable parameter, deriving free energy without analytical continuation, and demonstrates improved retrieval performance through simulations.

Findings

Negative n values lead to pattern decorrelation and interference reduction.

Partial annealing enhances retrieval in biased and complex pattern regimes.

Simulation confirms improved memory organization and retrieval performance.

Abstract

Using the Hopfield model as a benchmark case, the present work focuses on the investigation of partially annealed associative neural networks, wherein neural dynamics is coupled to slowly evolving patterns within the two-temperature-two-timescale framework. This setting inherently introduces a real parameter n, reminiscent of the number of replicas in the celebrated replica trick, that tunes the separation of timescales and the effective interaction between fast (i.e. the neurons) and slow (i.e. the synapses) degrees of freedom. By adapting Guerra's interpolation to the case, we derive the free energy without relying on analytical continuation. The obtained results demonstrate that negative values of n induce a progressive decorrelation of the stored patterns, thereby effectively reducing interference, promoting orthogonal configurations and ultimately conferring to the network the maximal storage alphac=1. Numerical simulations based on a mean field Monte Carlo dynamics have been employed to confirm this scenario and prove that partial annealing restores retrieval in challenging regimes, such as in the presence of biased patterns, outperforming standard decorrelation methods. These findings underscore the notion of partial annealing as an adaptive mechanism for enhancing memory organisation and retrieval in complex systems.