Inferring Concepts from Noisy Examples in Hopfield-like Neural Networks

TL;DR

This paper explores a variant of Hopfield neural networks that can infer archetypal concepts from limited noisy examples, revealing complex state behaviors and stronger RSB effects, enhancing understanding of memory and generalization.

Contribution

It introduces a modified pseudo-inverse learning rule for Hopfield networks and analyzes its complex state structure and generalization capabilities using mean-field replica theory.

Findings

Multiple coexisting states with diverse properties

Transitions between states can be smooth or abrupt

Stronger Replica Symmetry Breaking effects than standard models

Abstract

We study a variant of the pseudo-inverse learning rule for Hopfield-like Neural Networks, which allows the network to infer archetypal concepts on the basis of a limited number of examples. The mean-field replica theory for this model reveals how this generalization ability is mediated by a multitude of states, with diverse thermodynamic properties, coexisting with the standard Hopfield ones. They appear and vanish through smooth transitions or discontinuous jumps and, interestingly, show much stronger Replica Symmetry Breaking (RSB) effects than the standard Hopfield model, as captured by our 1RSB analysis. Our results, in excellent agreement with numerical simulations, provide deeper insight into the interplay between memory storage and generalization in attractor neural networks.