Path classification by stochastic linear recurrent neural networks

TL;DR

This paper analyzes stochastic linear recurrent neural networks for path classification, providing theoretical generalization bounds, demonstrating their robustness and trainability, and exploring the trade-off between accuracy and robustness.

Contribution

It introduces a theoretical framework for understanding stochastic RNNs in classification, including generalization bounds and robustness properties, supported by numerical experiments.

Findings

RNNs retain path signatures as the key information for classification

Theoretical generalization error bounds are established for stochastic RNNs

Numerical experiments confirm robustness and reveal a trade-off between accuracy and robustness

Abstract

We investigate the functioning of a classifying biological neural network from the perspective of statistical learning theory, modelled, in a simplified setting, as a continuous-time stochastic recurrent neural network (RNN) with identity activation function. In the purely stochastic (robust) regime, we give a generalisation error bound that holds with high probability, thus showing that the empirical risk minimiser is the best-in-class hypothesis. We show that RNNs retain a partial signature of the paths they are fed as the unique information exploited for training and classification tasks. We argue that these RNNs are easy to train and robust and back these observations with numerical experiments on both synthetic and real data. We also exhibit a trade-off phenomenon between accuracy and robustness.