Signature Reconstruction from Randomized Signatures

TL;DR

This paper investigates the extent to which signature features of curves can be reconstructed from controlled ordinary differential equations driven by random vector fields, revealing exponential growth in reconstructible features with neural network depth.

Contribution

It provides a theoretical analysis of the reconstructibility of signature features from controlled ODEs with random neural vector fields, extending Lie algebra results to machine learning.

Findings

Number of reconstructible signature features grows exponentially with neural network depth.

A linear independence condition on vector fields guarantees reconstruction of features.

Connects algebraic Lie theory with practical feature reconstruction in machine learning.

Abstract

Controlled ordinary differential equations driven by continuous bounded variation curves can be considered a continuous time analogue of recurrent neural networks for the construction of expressive features of the input curves. We ask up to which extent well known signature features of such curves can be reconstructed from controlled ordinary differential equations with (untrained) random vector fields. The answer turns out to be algebraically involved, but essentially the number of signature features, which can be reconstructed from the non-linear flow of the controlled ordinary differential equation, is exponential in its hidden dimension, when the vector fields are chosen to be neural with depth two. Moreover, we characterize a general linear independence condition on arbitrary vector fields, under which the signature features up to some fixed order can always be reconstructed. Algebraically speaking this complements in a quantitative manner several well known results from the theory of Lie algebras of vector fields and puts them in a context of machine learning.