Tolerance versus synaptic noise in dense associative memories

TL;DR

This paper investigates how synaptic and slow noise affect the retrieval capabilities of dense associative neural networks, revealing that higher-order interactions can tolerate more synaptic noise under certain conditions.

Contribution

It analyzes the interplay of slow and synaptic noise in dense associative memories, especially for p-plet interactions, using the duality with restricted Boltzmann machines.

Findings

For p=2, synaptic noise destroys retrieval as memory number scales with network size.

For p>2, synaptic noise is tolerated up to a certain bound in low-load regimes.

Higher-order interactions improve noise tolerance in associative memories.

Abstract

The retrieval capabilities of associative neural networks can be impaired by different kinds of noise: the fast noise (which makes neurons more prone to failure), the slow noise (stemming from interference among stored memories), and synaptic noise (due to possible flaws during the learning or the storing stage). In this work we consider dense associative neural networks, where neurons can interact in $p$-plets, in the absence of fast noise, and we investigate the interplay of slow and synaptic noise. In particular, leveraging on the duality between associative neural networks and restricted Boltzmann machines, we analyze the effect of corrupted information, imperfect learning and storing errors. For $p=2$ (corresponding to the Hopfield model) any source of synaptic noise breaks-down retrieval if the number of memories $K$ scales as the network size. For $p>2$, in the relatively low-load regime $K \sim N$, synaptic noise is tolerated up to a certain bound, depending on the density of the structure.