Non-linear motor control by local learning in spiking neural networks

TL;DR

This paper introduces FOLLOW, a local learning rule for spiking neural networks that enables online control of non-linear body dynamics by learning an inverse model of the system.

Contribution

The paper presents a novel local, stable, supervised learning scheme for spiking neural networks to control non-linear dynamics, demonstrated on a two-link arm.

Findings

The FOLLOW algorithm successfully trains a spiking neural network to control a non-linear arm.

The differential feedforward architecture yields the lowest test error.

The learned inverse model accurately reproduces desired trajectories.

Abstract

Learning weights in a spiking neural network with hidden neurons, using local, stable and online rules, to control non-linear body dynamics is an open problem. Here, we employ a supervised scheme, Feedback-based Online Local Learning Of Weights (FOLLOW), to train a network of heterogeneous spiking neurons with hidden layers, to control a two-link arm so as to reproduce a desired state trajectory. The network first learns an inverse model of the non-linear dynamics, i.e. from state trajectory as input to the network, it learns to infer the continuous-time command that produced the trajectory. Connection weights are adjusted via a local plasticity rule that involves pre-synaptic firing and post-synaptic feedback of the error in the inferred command. We choose a network architecture, termed differential feedforward, that gives the lowest test error from different feedforward and recurrent architectures. The learned inverse model is then used to generate a continuous-time motor command to control the arm, given a desired trajectory.