An Entropic Associative Memory

TL;DR

This paper introduces an entropic associative memory model based on Relational-Indeterminate Computing, capable of storing and retrieving symbolic and distributed representations, mimicking natural memory properties with adjustable entropy for optimal performance.

Contribution

It presents a novel associative memory architecture that combines symbolic and distributed features using entropy-controlled registers, addressing limitations of previous models.

Findings

Memory performance depends on entropy levels, with an optimal range for recognition and recall.

The model successfully simulates visual memory for handwritten digits.

Memory operations could be highly efficient on parallel architectures.

Abstract

Natural memories are associative, declarative and distributed. Symbolic computing memories resemble natural memories in their declarative character, and information can be stored and recovered explicitly; however, they lack the associative and distributed properties of natural memories. Sub-symbolic memories developed within the connectionist or artificial neural networks paradigm are associative and distributed, but are unable to express symbolic structure and information cannot be stored and retrieved explicitly; hence, they lack the declarative property. To address this dilemma, we use Relational-Indeterminate Computing to model associative memory registers that hold distributed representations of individual objects. This mode of computing has an intrinsic computing entropy which measures the indeterminacy of representations. This parameter determines the operational characteristics of the memory. Associative registers are embedded in an architecture that maps concrete images expressed in modality-specific buffers into abstract representations, and vice versa, and the memory system as a whole fulfills the three properties of natural memories. The system has been used to model a visual memory holding the representations of hand-written digits, and recognition and recall experiments show that there is a range of entropy values, not too low and not too high, in which associative memory registers have a satisfactory performance. The similarity between the cue and the object recovered in memory retrieve operations depends on the entropy of the memory register holding the representation of the corresponding object. The experiments were implemented in a simulation using a standard computer, but a parallel architecture may be built where the memory operations would take a very reduced number of computing steps.