Eigenvector Dreaming

TL;DR

This paper provides an analytical understanding of Hebbian Unlearning in Hopfield networks by describing its effects on the spectrum and eigenvectors of the coupling matrix, leading to new, more transparent dreaming algorithms.

Contribution

It introduces a spectral analysis framework for Hebbian Unlearning and proposes novel dreaming algorithms that are both computationally effective and analytically transparent.

Findings

Spectral evolution explains Hebbian Unlearning effects.

New algorithms outperform original schemes in effectiveness.

Analytical models improve understanding of associative memory enhancement.

Abstract

Among the performance-enhancing procedures for Hopfield-type networks that implement associative memory, Hebbian Unlearning (or dreaming) strikes for its simplicity and its clear biological interpretation. Yet, it does not easily lend itself to a clear analytical understanding. Here we show how Hebbian Unlearning can be effectively described in terms of a simple evolution of the spectrum and the eigenvectors of the coupling matrix. We use these ideas to design new dreaming algorithms that are effective from a computational point of view, and are analytically far more transparent than the original scheme.