Prediction of Dynamical time Series Using Kernel Based Regression and Smooth Splines

TL;DR

This paper demonstrates that combining smooth splines with kernel-based regression significantly improves the accuracy of dynamical time series prediction and noise removal, especially when operating on delay coordinate data.

Contribution

It introduces a novel approach that combines smooth splines with kernel regression for better denoising and prediction of dynamical systems, with proven convergence guarantees.

Findings

Combination improves prediction accuracy by a factor of 2 or more.

Convergence to the exact predictor is proven for smooth flows.

Outperforms existing methods by up to 100 times in some cases.

Abstract

Prediction of dynamical time series with additive noise using support vector machines or kernel based regression has been proved to be consistent for certain classes of discrete dynamical systems. Consistency implies that these methods are effective at computing the expected value of a point at a future time given the present coordinates. However, the present coordinates themselves are noisy, and therefore, these methods are not necessarily effective at removing noise. In this article, we consider denoising and prediction as separate problems for flows, as opposed to discrete time dynamical systems, and show that the use of smooth splines is more effective at removing noise. Combination of smooth splines and kernel based regression yields predictors that are more accurate on benchmarks typically by a factor of 2 or more. We prove that kernel based regression in combination with smooth splines converges to the exact predictor for time series extracted from any compact invariant set of any sufficiently smooth flow. As a consequence of convergence, one can find examples where the combination of kernel based regression with smooth splines is superior by even a factor of $100$. The predictors that we compute operate on delay coordinate data and not the full state vector, which is typically not observable.