Unsupervised prototype learning in an associative-memory network

TL;DR

This paper explores unsupervised learning in a generalized Hopfield network, showing its equivalence to a semi-restricted Boltzmann machine, and introduces a spectral method for extracting core data prototypes.

Contribution

It demonstrates the equivalence of Hopfield networks to semi-restricted Boltzmann machines and proposes a spectral method for extracting data prototypes.

Findings

Hopfield network can learn faithful internal representations.

Proposed spectral method effectively extracts core data prototypes.

Hopfield model serves as a building block for deep learning systems.

Abstract

Unsupervised learning in a generalized Hopfield associative-memory network is investigated in this work. First, we prove that the (generalized) Hopfield model is equivalent to a semi-restricted Boltzmann machine with a layer of visible neurons and another layer of hidden binary neurons, so it could serve as the building block for a multilayered deep-learning system. We then demonstrate that the Hopfield network can learn to form a faithful internal representation of the observed samples, with the learned memory patterns being prototypes of the input data. Furthermore, we propose a spectral method to extract a small set of concepts (idealized prototypes) as the most concise summary or abstraction of the empirical data.