Inference and learning in sparse systems with multiple states

TL;DR

This paper develops a cavity method-based inference approach for sparse systems with multiple states, enabling learning protocols that store patterns without entering a spin glass phase.

Contribution

It introduces a novel cavity method for inference in non-ergodic phases of sparse systems and proposes a local learning protocol for neural networks.

Findings

Effective inference from data sampled from attractor states.

A simple local learning protocol that avoids spin glass phase.

Ability to store patterns with finite overlap without memory loss.

Abstract

We discuss how inference can be performed when data are sampled from the non-ergodic phase of systems with multiple attractors. We take as model system the finite connectivity Hopfield model in the memory phase and suggest a cavity method approach to reconstruct the couplings when the data are separately sampled from few attractor states. We also show how the inference results can be converted into a learning protocol for neural networks in which patterns are presented through weak external fields. The protocol is simple and fully local, and is able to store patterns with a finite overlap with the input patterns without ever reaching a spin glass phase where all memories are lost.