A fast memoryless predictive algorithm in a chain of recurrent neural networks

TL;DR

This paper extends a fast, memoryless prediction algorithm from a single recurrent neural network to a chain of RNs, enabling sequence prediction without storing inputs and increasing robustness, with implications for neural systems.

Contribution

It introduces a generalized, memoryless predictive algorithm for chains of recurrent neural networks, applicable in natural neural systems and improving robustness over traditional methods.

Findings

Algorithm can predict sequences without input storage

Applicable to chains of RNs, not just single networks

Enhances robustness compared to moving window methods

Abstract

In the recent publication (arxiv:2007.08063v2 [cs.LG]) a fast prediction algorithm for a single recurrent network (RN) was suggested. In this manuscript we generalize this approach to a chain of RNs and show that it can be implemented in natural neural systems. When the network is used recursively to predict sequence of values the proposed algorithm does not require to store the original input sequence. It increases robustness of the new approach compared to the standard moving/expanding window predictive procedure. We consider requirements on trained networks that allow to implement the proposed algorithm and discuss them in the neuroscience context.