Stochastic transitions of attractors in associative memory models with correlated noise

TL;DR

This paper explores how correlated noise influences state transitions in recurrent neural networks with associative memory, revealing that correlated noise can induce stochastic transitions between memory states and mixture states.

Contribution

It introduces a macroscopic dynamic framework to analyze the effects of correlated noise on neural network state transitions, extending previous models that focused on thermal noise.

Findings

Correlated noise can induce stochastic transitions between memory states.

Theoretical results agree with computer simulations.

Correlated noise influences network behavior more significantly than thermal noise.

Abstract

We investigate dynamics of recurrent neural networks with correlated noise to analyze the noise's effect. The mechanism of correlated firing has been analyzed in various models, but its functional roles have not been discussed in sufficient detail. Aoyagi and Aoki have shown that the state transition of a network is invoked by synchronous spikes. We introduce two types of noise to each neuron: thermal independent noise and correlated noise. Due to the effects of correlated noise, the correlation between neural inputs cannot be ignored, so the behavior of the network has sample dependence. We discuss two types of associative memory models: one with auto- and weak cross-correlation connections and one with hierarchically correlated patterns. The former is similar in structure to Aoyagi and Aoki's model. We show that stochastic transition can be presented by correlated rather than thermal noise. In the latter, we show stochastic transition from a memory state to a mixture state using correlated noise. To analyze the stochastic transitions, we derive a macroscopic dynamic description as a recurrence relation form of a probability density function when the correlated noise exists. Computer simulations agree with theoretical results.