Non-Convex Multi-species Hopfield models

TL;DR

This paper introduces a multi-species Hopfield model that incorporates layered interactions and is solvable in low-load regimes, bridging associative memory models with modern neural network architectures.

Contribution

It generalizes the Hopfield model to multiple species with intra- and inter-group interactions, including special cases like RBM and BAM.

Findings

Model is exactly solvable in low-load regime.

Includes special cases like RBM and BAM.

Handles non-definite quadratic Hamiltonians.

Abstract

In this work we introduce a multi-species generalization of the Hopfield model for associative memory, where neurons are divided into groups and both inter-groups and intra-groups pair-wise interactions are considered, with different intensities. Thus, this system contains two of the main ingredients of modern Deep neural network architectures: Hebbian interactions to store patterns of information and multiple layers coding different levels of correlations. The model is completely solvable in the low-load regime with a suitable generalization of the Hamilton-Jacobi technique, despite the Hamiltonian can be a non-definite quadratic form of the magnetizations. The family of multi-species Hopfield model includes, as special cases, the 3-layers Restricted Boltzmann Machine (RBM) with Gaussian hidden layer and the Bidirectional Associative Memory (BAM) model.