Competitive learning to generate sparse representations for associative memory

TL;DR

This paper introduces a biologically plausible neural network that encodes images into sparse representations suitable for associative memory, outperforming baseline methods and approaching optimal random code performance.

Contribution

The authors propose a new competitive learning network that generates sparse codes for images, enabling effective associative memory with biological plausibility.

Findings

Outperforms sparse coding baselines in experiments

Approaches the performance of optimal random codes

Works effectively on visual datasets

Abstract

One of the most well established brain principles, hebbian learning, has led to the theoretical concept of neural assemblies. Based on it, many interesting brain theories have spawned. Palm's work implements this concept through binary associative memory, in a model that not only has a wide cognitive explanatory power but also makes neuroscientific predictions. Yet, associative memory can only work with logarithmic sparse representations, which makes it extremely difficult to apply the model to real data. We propose a biologically plausible network that encodes images into codes that are suitable for associative memory. It is organized into groups of neurons that specialize on local receptive fields, and learn through a competitive scheme. After conducting auto- and hetero-association experiments on two visual data sets, we can conclude that our network not only beats sparse coding baselines, but also that it comes close to the performance achieved using optimal random codes.