Prime and Modulate Learning: Generation of forward models with signed back-propagation and environmental cues

TL;DR

This paper introduces Prime and Modulate learning, a novel method for training neural networks that uses error sign priming and environmental cues to improve convergence speed without normalization.

Contribution

It proposes a new learning paradigm inspired by neuromodulation, enhancing forward model learning by priming with error signs and modulating with environmental relevance signals.

Findings

Sign-based priming improves convergence speed

Environmental cues enrich learning process

Method outperforms traditional back-propagation in real-time robotic tasks

Abstract

Deep neural networks employing error back-propagation for learning can suffer from exploding and vanishing gradient problems. Numerous solutions have been proposed such as normalisation techniques or limiting activation functions to linear rectifying units. In this work we follow a different approach which is particularly applicable to closed-loop learning of forward models where back-propagation makes exclusive use of the sign of the error signal to prime the learning, whilst a global relevance signal modulates the rate of learning. This is inspired by the interaction between local plasticity and a global neuromodulation. For example, whilst driving on an empty road, one can allow for slow step-wise optimisation of actions, whereas, at a busy junction, an error must be corrected at once. Hence, the error is the priming signal and the intensity of the experience is a modulating factor in the weight change. The advantages of this Prime and Modulate paradigm is twofold: it is free from normalisation and it makes use of relevant cues from the environment to enrich the learning. We present a mathematical derivation of the learning rule in z-space and demonstrate the real-time performance with a robotic platform. The results show a significant improvement in the speed of convergence compared to that of the conventional back-propagation.