Generalising E-prop to Deep Networks

TL;DR

This paper extends the E-prop learning algorithm to deep recurrent networks, enabling online training across multiple layers and time without backpropagation through time, thus making biologically plausible learning in deep networks feasible.

Contribution

It introduces a novel recursion for eligibility traces that allows E-prop to be applied to arbitrarily deep recurrent networks, enabling online, layer-wise credit assignment.

Findings

E-prop can be extended to deep networks with a new depth recursion.

The method enables accurate online learning across time and depth.

It eliminates the need for backpropagation through time in deep recurrent training.

Abstract

Recurrent networks are typically trained with backpropagation through time (BPTT). However, BPTT requires storing the history of all states in the network and then replaying them sequentially backwards in time. This computation appears extremely implausible for the brain to implement. Real Time Recurrent Learning (RTRL) proposes an mathematically equivalent alternative where gradient information is propagated forwards in time locally alongside the regular forward pass, however it has significantly greater computational complexity than BPTT which renders it impractical for large networks. E-prop proposes an approximation of RTRL which reduces its complexity to the level of BPTT while maintaining a purely online forward update which can be implemented by an eligibility trace at each synapse. However, works on RTRL and E-prop ubiquitously investigate learning in a single layer with recurrent dynamics. However, learning in the brain spans multiple layers and consists of both hierarchal dynamics in depth as well as time. In this mathematical note, we extend the E-prop framework to handle arbitrarily deep networks, deriving a novel recursion relationship across depth which extends the eligibility traces of E-prop to deeper layers. Our results thus demonstrate an online learning algorithm can perform accurate credit assignment across both time and depth simultaneously, allowing the training of deep recurrent networks without backpropagation through time.