Biologically-plausible backpropagation through arbitrary timespans via local neuromodulators

TL;DR

This paper introduces ModProp, a biologically plausible method for temporal credit assignment in neural networks, utilizing neuromodulators to propagate credit signals over arbitrary timespans, outperforming existing rules.

Contribution

Proposes ModProp, a novel biologically plausible gradient approximation method that uses neuromodulators to propagate credit over arbitrary time horizons in recurrent networks.

Findings

ModProp outperforms existing biologically-plausible temporal credit assignment rules.

Simulation results demonstrate ModProp's effectiveness on benchmark temporal tasks.

In-silico implementation of ModProp offers a low-complexity alternative to backpropagation through time.

Abstract

The spectacular successes of recurrent neural network models where key parameters are adjusted via backpropagation-based gradient descent have inspired much thought as to how biological neuronal networks might solve the corresponding synaptic credit assignment problem. There is so far little agreement, however, as to how biological networks could implement the necessary backpropagation through time, given widely recognized constraints of biological synaptic network signaling architectures. Here, we propose that extra-synaptic diffusion of local neuromodulators such as neuropeptides may afford an effective mode of backpropagation lying within the bounds of biological plausibility. Going beyond existing temporal truncation-based gradient approximations, our approximate gradient-based update rule, ModProp, propagates credit information through arbitrary time steps. ModProp suggests that modulatory signals can act on receiving cells by convolving their eligibility traces via causal, time-invariant and synapse-type-specific filter taps. Our mathematical analysis of ModProp learning, together with simulation results on benchmark temporal tasks, demonstrate the advantage of ModProp over existing biologically-plausible temporal credit assignment rules. These results suggest a potential neuronal mechanism for signaling credit information related to recurrent interactions over a longer time horizon. Finally, we derive an in-silico implementation of ModProp that could serve as a low-complexity and causal alternative to backpropagation through time.