Log Neural Controlled Differential Equations: The Lie Brackets Make a Difference

TL;DR

Log-NCDEs introduce a novel approach using Lie brackets and Log-ODEs to enhance neural controlled differential equations, significantly improving performance on complex multivariate time series data.

Contribution

The paper presents Log-NCDEs, a new method that leverages Lie brackets and the Log-ODE technique to improve training and accuracy of neural CDEs.

Findings

Log-NCDEs outperform existing models like NCDEs and NRDEs.

The method is effective on large, irregularly sampled multivariate time series.

Log-NCDEs achieve state-of-the-art results on multiple datasets.

Abstract

The vector field of a controlled differential equation (CDE) describes the relationship between a control path and the evolution of a solution path. Neural CDEs (NCDEs) treat time series data as observations from a control path, parameterise a CDE's vector field using a neural network, and use the solution path as a continuously evolving hidden state. As their formulation makes them robust to irregular sampling rates, NCDEs are a powerful approach for modelling real-world data. Building on neural rough differential equations (NRDEs), we introduce Log-NCDEs, a novel, effective, and efficient method for training NCDEs. The core component of Log-NCDEs is the Log-ODE method, a tool from the study of rough paths for approximating a CDE's solution. Log-NCDEs are shown to outperform NCDEs, NRDEs, the linear recurrent unit, S5, and MAMBA on a range of multivariate time series datasets with up to $50{,}000$ observations.