Thouless-Anderson-Palmer equation for analog neural network with temporally fluctuating white synaptic noise

TL;DR

This paper derives the Thouless-Anderson-Palmer (TAP) equation for analog neural networks with temporally fluctuating white synaptic noise, enabling analysis of stochastic networks without an energy function using the self-consistent signal-to-noise analysis.

Contribution

It introduces a hybrid method combining the cavity method and SCSNA to derive the TAP equation for stochastic neural networks with synaptic noise.

Findings

Derived the TAP equation for networks with synaptic noise

Established a set of order parameter equations for such networks

Provided a framework for analyzing stochastic neural networks without energy functions

Abstract

Effects of synaptic noise on the retrieval process of associative memory neural networks are studied from the viewpoint of neurobiological and biophysical understanding of information processing in the brain. We investigate the statistical mechanical properties of stochastic analog neural networks with temporally fluctuating synaptic noise, which is assumed to be white noise. Such networks, in general, defy the use of the replica method, since they have no energy concept. The self-consistent signal-to-noise analysis (SCSNA), which is an alternative to the replica method for deriving a set of order parameter equations, requires no energy concept and thus becomes available in studying networks without energy functions. Applying the SCSNA to stochastic network requires the knowledge of the Thouless-Anderson-Palmer (TAP) equation which defines the deterministic networks equivalent to the original stochastic ones. The study of the TAP equation which is of particular interest for the case without energy concept is very few, while it is closely related to the SCSNA in the case with energy concept. This paper aims to derive the TAP equation for networks with synaptic noise together with a set of order parameter equations by a hybrid use of the cavity method and the SCSNA.