A correspondence between Hebbian unlearning and steady states generated by nonequilibrium dynamics

TL;DR

This paper reveals a connection between nonequilibrium dynamics in associative memory models and Hebbian unlearning, showing how steady states achieved through these dynamics resemble those from traditional unlearning methods, potentially enhancing memory systems.

Contribution

The study analytically and computationally demonstrates that nonequilibrium dynamics produce steady states similar to Hebbian unlearning, offering a new perspective on memory encoding and recall.

Findings

Steady states from nonequilibrium dynamics resemble Hebbian unlearning states.

Nonequilibrium dynamics can increase memory storage capacity.

The work bridges dynamical systems and classical unlearning techniques.

Abstract

The classic paradigms for learning and memory recall focus on strengths of synaptic couplings and how these can be modulated to encode memories. In a previous paper [A. K. Behera, M. Rao, S. Sastry, and S. Vaikuntanathan, Physical Review X 13, 041043 (2023)], we demonstrated how a specific non-equilibrium modification of the dynamics of an associative memory system can lead to increase in storage capacity. In this work, using analytical theory and computational inference schemes, we show that the dynamical steady state accessed is in fact similar to those accessed after the operation of a classic unsupervised scheme for improving memory recall, Hebbian unlearning or ``dreaming". Together, our work suggests how nonequilibrium dynamics can provide an alternative route for controlling the memory encoding and recall properties of a variety of synthetic (neuromorphic) and biological systems.