Full solution for the storage of correlated memories in an autoassociative memory

TL;DR

This paper presents a comprehensive solution for storing correlated memories in an autoassociative network, improving capacity through various modifications and matching experimental data on semantic memory patterns.

Contribution

It provides the full, non-diluted solution for stable states in correlated pattern storage, extending previous work and demonstrating capacity enhancements via synaptic modulation techniques.

Findings

The model accurately fits semantic memory pattern data.

Capacity can be optimized with popularity-modulated Hebbian learning.

Storage capacity scales with pattern information content.

Abstract

We complement our previous work [arxiv: 0707.0565] with the full (non diluted) solution describing the stable states of an attractor network that stores correlated patterns of activity. The new solution provides a good fit of simulations of a network storing the feature norms of McRae and colleagues [McRae et al, 2005], experimentally obtained combinations of features representing concepts in semantic memory. We discuss three ways to improve the storage capacity of the network: adding uninformative neurons, removing informative neurons and introducing popularity-modulated hebbian learning. We show that if the strength of synapses is modulated by an exponential decay of the popularity of the pre-synaptic neuron, any distribution of patterns can be stored and retrieved with approximately an optimal storage capacity - i.e, C ~ I.p, the minimum number of connections per neuron needed to sustain the retrieval of a pattern is proportional to the information content of the pattern multiplied by the number of patterns stored in the network.