Benchmarking optimality of time series classification methods in distinguishing diffusions

TL;DR

This paper introduces a benchmarking framework using the likelihood ratio test to evaluate the optimality of various time series classification algorithms in distinguishing diffusion processes, highlighting their strengths and limitations.

Contribution

It proposes an efficient, likelihood ratio test-based benchmark for TSC algorithms, assessing their optimality in diffusion process classification without training.

Findings

Random forest, ResNet, and ROCKET achieve LRT optimality for univariate and Gaussian processes.

These algorithms are suboptimal for high-dimensional nonlinear multivariate series.

The benchmark analyzes accuracy dependence on series length, dimension, and sampling frequency.

Abstract

Statistical optimality benchmarking is crucial for analyzing and designing time series classification (TSC) algorithms. This study proposes to benchmark the optimality of TSC algorithms in distinguishing diffusion processes by the likelihood ratio test (LRT). The LRT is an optimal classifier by the Neyman-Pearson lemma. The LRT benchmarks are computationally efficient because the LRT does not need training, and the diffusion processes can be efficiently simulated and are flexible to reflect the specific features of real-world applications. We demonstrate the benchmarking with three widely-used TSC algorithms: random forest, ResNet, and ROCKET. These algorithms can achieve the LRT optimality for univariate time series and multivariate Gaussian processes. However, these model-agnostic algorithms are suboptimal in classifying high-dimensional nonlinear multivariate time series. Additionally, the LRT benchmark provides tools to analyze the dependence of classification accuracy on the time length, dimension, temporal sampling frequency, and randomness of the time series.