LORD: Lower-Dimensional Embedding of Log-Signature in Neural Rough Differential Equations

TL;DR

This paper introduces LORD, a method that embeds higher-depth log-signature features into lower-depth ones within neural rough differential equations, significantly improving processing efficiency and accuracy for long time-series data.

Contribution

We propose LORD, an autoencoder-based approach that reduces the dimensionality of log-signature transforms, stabilizing training and enhancing model performance on long time-series.

Findings

Up to 75% improvement in classification and forecasting metrics

Enhanced stability in training neural rough differential equations

Effective embedding of higher-depth log-signature knowledge into lower-depth features

Abstract

The problem of processing very long time-series data (e.g., a length of more than 10,000) is a long-standing research problem in machine learning. Recently, one breakthrough, called neural rough differential equations (NRDEs), has been proposed and has shown that it is able to process such data. Their main concept is to use the log-signature transform, which is known to be more efficient than the Fourier transform for irregular long time-series, to convert a very long time-series sample into a relatively shorter series of feature vectors. However, the log-signature transform causes non-trivial spatial overheads. To this end, we present the method of LOweR-Dimensional embedding of log-signature (LORD), where we define an NRDE-based autoencoder to implant the higher-depth log-signature knowledge into the lower-depth log-signature. We show that the encoder successfully combines the higher-depth and the lower-depth log-signature knowledge, which greatly stabilizes the training process and increases the model accuracy. In our experiments with benchmark datasets, the improvement ratio by our method is up to 75\% in terms of various classification and forecasting evaluation metrics.