A biological gradient descent for prediction through a combination of STDP and homeostatic plasticity

TL;DR

This paper presents a mathematical formalism showing how combined biological mechanisms, STDP and homeostatic plasticity, enable recurrent neural networks to perform online gradient descent for prediction without bifurcation issues.

Contribution

It introduces a novel formalism combining STDP and homeostatic plasticity, demonstrating their role in enabling neural networks to predict stimuli through gradient descent.

Findings

Networks implement online gradient descent of stimulus distance.

Convergence to equilibrium allows spontaneous stimulus reproduction.

No bifurcation issues occur during learning in these networks.

Abstract

Identifying, formalizing and combining biological mechanisms which implement known brain functions, such as prediction, is a main aspect of current research in theoretical neuroscience. In this letter, the mechanisms of Spike Timing Dependent Plasticity (STDP) and homeostatic plasticity, combined in an original mathematical formalism, are shown to shape recurrent neural networks into predictors. Following a rigorous mathematical treatment, we prove that they implement the online gradient descent of a distance between the network activity and its stimuli. The convergence to an equilibrium, where the network can spontaneously reproduce or predict its stimuli, does not suffer from bifurcation issues usually encountered in learning in recurrent neural networks.