Dense Associative Memory in a Nonlinear Optical Hopfield Neural Network

TL;DR

This paper introduces a nonlinear optical Hopfield neural network system for Dense Associative Memories, demonstrating significant improvements in pattern storage capacity and retrieval quality, with potential applications in big-data and computer vision tasks.

Contribution

First experimental implementation of a nonlinear optical Hopfield neural network for Dense Associative Memories with enhanced capacity and robustness.

Findings

Ten-fold increase in uncorrelated pattern storage capacity.

Up to 50 times more patterns stored for correlated patterns.

5.5 times improvement in pattern retrieval quality.

Abstract

Modern Hopfield Neural Networks (HNNs), also known as Dense Associative Memories (DAMs), enhance the performance of simple recurrent neural networks by leveraging the nonlinearities in their energy functions. They have broad applications in combinatorial optimization, high-capacity memory storage, deep learning transformers, and correlated pattern recognition. Thus far, research on DAMs has been primarily theoretical, with implementations limited to CPUs and GPUs. In this work, for the first time to our knowledge, we propose and experimentally demonstrate a nonlinear optical Hopfield neural network (NOHNN) system for realizing DAMs using correlated patterns. Our NOHNN incorporates effective 2-body and 4-body interactions in its energy function. The inclusion of 4-body interaction scores a minimum ten-fold improvement in the number of uncorrelated patterns that can be stored and retrieved, significantly surpassing the traditional capacity limit for traditional HNNs. For correlated patterns, depending on their average correlation, up to 50 times more patterns can be stored compared to traditional HNNs. To test the system's robustness, the benchmark testing is performed on MNIST handwritten digit patterns. The results show a 5.5 times improvement in the pattern storage along with the retrieval of cleaner and less noisy patterns. These results highlight the potential of nonlinear optical DAMs for practical applications in challenging big-data optimization, computer vision and graph network tasks.