Benchmarking local Hebbian learning rules for memory storage and prototype extraction

TL;DR

This paper benchmarks seven Hebbian learning rules in neural networks for associative memory and prototype extraction, evaluating their capacity, robustness, and sensitivity to data correlations.

Contribution

It provides a comprehensive comparison of Hebbian learning rules for memory storage and prototype extraction in neural networks.

Findings

Bayesian-Hebbian rules show highest capacity across conditions.

Additive Hebb rule has the worst storage capacity.

Covariance learning is robust but has moderate capacity.

Abstract

Associative memory or content-addressable memory is an important component function in computer science and information processing, and at the same time a key concept in cognitive and computational brain science. Many different neural network architectures and learning rules have been proposed to model the brain's associative memory while investigating key component functions like figure-ground segmentation, perceptual reconstruction and rivalry. A less investigated but equally important capability of associative memory is prototype extraction where the training set comprises distorted prototype instances and the task is to recall the correct generating prototype given a new distorted instance. In this paper we benchmark associative memory function of seven different Hebbian learning rules employed in non-modular and modular recurrent networks with winner-take-all dynamics operating on moderately sparse binary patterns. We measure pattern storage and weight information capacity, prototype extraction capabilities, and sensitivity to correlations in data. The original additive Hebb rule comes out with worst capacity, covariance learning proves to be robust but with moderate capacity, and the Bayesian-Hebbian learning rules show highest capacity in almost all different conditions tested.