Classification and Generation of real-world data with an Associative Memory Model

TL;DR

This paper extends the Willshaw Associative Memory Model to a multi-modality framework, enabling it to store, retrieve, classify, and generate real-world data like images and labels, inspired by biological memory processes.

Contribution

It introduces a multi-modality associative memory model with an iterative retrieval algorithm, enhancing capabilities for data completion, classification, and generation on the MNIST dataset.

Findings

Successfully stored and retrieved visual and textual data

Achieved pattern completion and classification with a single memory

Demonstrated potential for biologically-inspired learning tasks

Abstract

Drawing from memory the face of a friend you have not seen in years is a difficult task. However, if you happen to cross paths, you would easily recognize each other. The biological memory is equipped with an impressive compression algorithm that can store the essential, and then infer the details to match perception. The Willshaw Memory is a simple abstract model for cortical computations which implements mechanisms of biological memories. Using our recently proposed sparse coding prescription for visual patterns, this model can store and retrieve an impressive amount of real-world data in a fault-tolerant manner. In this paper, we extend the capabilities of the basic Associative Memory Model by using a Multiple-Modality framework. In this setting, the memory stores several modalities (e.g., visual, or textual) of each pattern simultaneously. After training, the memory can be used to infer missing modalities when just a subset is perceived. Using a simple encoder-memory-decoder architecture, and a newly proposed iterative retrieval algorithm for the Willshaw Model, we perform experiments on the MNIST dataset. By storing both the images and labels as modalities, a single Memory can be used not only to retrieve and complete patterns but also to classify and generate new ones. We further discuss how this model could be used for other learning tasks, thus serving as a biologically-inspired framework for learning.