A Mathematical Analysis of Memory Lifetime in a simple Network Model of Memory

TL;DR

This paper analyzes how a simple neural network model learns and forgets external signals over time, providing bounds on memory retention influenced by noise and network size.

Contribution

It introduces a finite-time analysis of memory lifetime in a neural network model using Markov chain dynamics, with bounds applicable to finite and large networks.

Findings

Derived a lower bound on the number of stimuli before forgetting occurs

Provided finite and asymptotic bounds on memory retention

Numerical illustrations support theoretical results

Abstract

We study the learning of an external signal by a neural network and the time to forget it when this network is submitted to noise. The presentation of an external stimulus to the recurrent network of binary neurons may change the state of the synapses. Multiple presentations of a unique signal leads to its learning. Then, during the forgetting time, the presentation of other signals (noise) may also modify the synaptic weights. We construct an estimator of the initial signal thanks to the synaptic currents and define by this way a probability of error. In our model, these synaptic currents evolve as Markov chains. We study the dynamics of these Markov chains and obtain a lower bound on the number of external stimuli that the network can receive before the initial signal is considered as forgotten (probability of error above a given threshold). Our results hold for finite size networks as well as in the large size asymptotic. Our results are based on a finite time analysis rather than large time asymptotic. We finally present numerical illustrations of our results.