Stochastic learning in a neural network with adapting synapses

TL;DR

This paper analyzes a neural network model with stochastic synaptic adaptation, deriving analytical equations for its macroscopic behavior in the large network limit, highlighting how learning and computation coexist with low synaptic transition probabilities.

Contribution

It provides an analytical framework for understanding stochastic learning in neural networks with multi-state synapses, applicable in the large network limit.

Findings

Analytical flow equations for neuron and synapse dynamics.

Negligible correlations in the large network limit.

Model captures simultaneous learning and computation.

Abstract

We consider a neural network with adapting synapses whose dynamics can be analitically computed. The model is made of $N$ neurons and each of them is connected to $K$ input neurons chosen at random in the network. The synapses are $n$-states variables which evolve in time according to Stochastic Learning rules; a parallel stochastic dynamics is assumed for neurons. Since the network maintains the same dynamics whether it is engaged in computation or in learning new memories, a very low probability of synaptic transitions is assumed. In the limit $N\to\infty$ with $K$ large and finite, the correlations of neurons and synapses can be neglected and the dynamics can be analitically calculated by flow equations for the macroscopic parameters of the system.