Maximum Likelihood Associative Memories

TL;DR

This paper analyzes associative memories based on maximum likelihood principles, deriving error rates, memory requirements, and computational complexity, and compares them with existing neural network architectures.

Contribution

It introduces a maximum likelihood framework for associative memories, providing bounds on error, memory, and complexity, and compares with existing models.

Findings

Derived minimum residual error rates for uniform binary data

Determined minimum memory requirements for data storage

Bounded computational complexity for message retrieval

Abstract

Associative memories are structures that store data in such a way that it can later be retrieved given only a part of its content -- a sort-of error/erasure-resilience property. They are used in applications ranging from caches and memory management in CPUs to database engines. In this work we study associative memories built on the maximum likelihood principle. We derive minimum residual error rates when the data stored comes from a uniform binary source. Second, we determine the minimum amount of memory required to store the same data. Finally, we bound the computational complexity for message retrieval. We then compare these bounds with two existing associative memory architectures: the celebrated Hopfield neural networks and a neural network architecture introduced more recently by Gripon and Berrou.