Slow synaptic dynamics in a network: from exponential to power-law forgetting

TL;DR

This paper models synaptic dynamics in neural networks, revealing how competition and critical points lead to power-law relaxation and the emergence of long- and short-term memory.

Contribution

It introduces a novel polarity-driven competition rule and demonstrates universal power-law relaxation behaviors near critical and tricritical points.

Findings

Power-law relaxation of synaptic strength at critical points

Emergence of long- and short-term memory from network dynamics

Universal $1/t$ and $1/\sqrt{t}$ relaxation behaviors

Abstract

We investigate a mean-field model of interacting synapses on a directed neural network. Our interest lies in the slow adaptive dynamics of synapses, which are driven by the fast dynamics of the neurons they connect. Cooperation is modelled from the usual Hebbian perspective, while competition is modelled by an original polarity-driven rule. The emergence of a critical manifold culminating in a tricritical point is crucially dependent on the presence of synaptic competition. This leads to a universal $1/t$ power-law relaxation of the mean synaptic strength along the critical manifold and an equally universal $1/\sqrt{t}$ relaxation at the tricritical point, to be contrasted with the exponential relaxation that is otherwise generic. In turn, this leads to the natural emergence of long- and short-term memory from different parts of parameter space in a synaptic network, which is the most novel and important result of our present investigations.