Learning by message-passing in networks of discrete synapses

TL;DR

This paper introduces a message-passing learning algorithm for networks with binary synapses, capable of storing nearly the maximum number of patterns, applicable to large, biologically relevant systems, and suitable for online, fault-tolerant learning.

Contribution

It presents a novel message-passing learning method that approaches information-theoretic limits in binary synapse networks, scalable to large sizes and adaptable for online learning.

Findings

Achieves near-saturation of information storage capacity.

Works across various network topologies and sizes.

Enables online, fault-tolerant learning protocols.

Abstract

We show that a message-passing process allows to store in binary "material" synapses a number of random patterns which almost saturates the information theoretic bounds. We apply the learning algorithm to networks characterized by a wide range of different connection topologies and of size comparable with that of biological systems (e.g. $n\simeq10^{5}-10^{6}$). The algorithm can be turned into an on-line --fault tolerant-- learning protocol of potential interest in modeling aspects of synaptic plasticity and in building neuromorphic devices.