Estimating Vector Fields from Noisy Time Series

TL;DR

This paper introduces a neural network approach with an alternating minimization scheme to accurately estimate vector fields from noisy time series data, improving robustness to high noise levels.

Contribution

It presents a neural shape function architecture and an alternating minimization method that enhance differential equation learning from noisy data, applicable in any finite dimension.

Findings

Retrofitting existing methods with the proposed scheme boosts noise robustness.

The neural shape function architecture retains approximation properties and interpretability.

The method achieves accurate vector field estimation even with 10% Gaussian noise.

Abstract

While there has been a surge of recent interest in learning differential equation models from time series, methods in this area typically cannot cope with highly noisy data. We break this problem into two parts: (i) approximating the unknown vector field (or right-hand side) of the differential equation, and (ii) dealing with noise. To deal with (i), we describe a neural network architecture consisting of tensor products of one-dimensional neural shape functions. For (ii), we propose an alternating minimization scheme that switches between vector field training and filtering steps, together with multiple trajectories of training data. We find that the neural shape function architecture retains the approximation properties of dense neural networks, enables effective computation of vector field error, and allows for graphical interpretability, all for data/systems in any finite dimension $d$. We also study the combination of either our neural shape function method or existing differential equation learning methods with alternating minimization and multiple trajectories. We find that retrofitting any learning method in this way boosts the method's robustness to noise. While in their raw form the methods struggle with 1% Gaussian noise, after retrofitting, they learn accurate vector fields from data with 10% Gaussian noise.