A Taxonomy of Recurrent Learning Rules

TL;DR

This paper clarifies the relationships among recurrent learning algorithms by deriving RTRL from BPTT, framing e-prop within this context, and introducing a family of algorithms that generalize e-prop.

Contribution

It provides a formal connection between RTRL, BPTT, and e-prop, and introduces a new family of algorithms extending e-prop.

Findings

Derived RTRL from BPTT with detailed notation

Framed e-prop as an approximation within this family

Introduced a new family of recurrent learning algorithms

Abstract

Backpropagation through time (BPTT) is the de facto standard for training recurrent neural networks (RNNs), but it is non-causal and non-local. Real-time recurrent learning is a causal alternative, but it is highly inefficient. Recently, e-prop was proposed as a causal, local, and efficient practical alternative to these algorithms, providing an approximation of the exact gradient by radically pruning the recurrent dependencies carried over time. Here, we derive RTRL from BPTT using a detailed notation bringing intuition and clarification to how they are connected. Furthermore, we frame e-prop within in the picture, formalising what it approximates. Finally, we derive a family of algorithms of which e-prop is a special case.