A Log-Linear Non-Parametric Online Changepoint Detection Algorithm based on Functional Pruning

TL;DR

This paper introduces NP-FOCuS, a nonparametric, log-linear computational cost online changepoint detection algorithm that outperforms existing methods in high-frequency data streams by leveraging functional pruning.

Contribution

The paper presents a novel nonparametric online changepoint detection method, NP-FOCuS, with efficient log-linear complexity suitable for high-frequency data, and demonstrates its superior detection power.

Findings

Outperforms existing nonparametric methods in various settings

Efficient log-linear computational complexity

Effective on both simulated and real high-frequency data

Abstract

Online changepoint detection aims to detect anomalies and changes in real-time in high-frequency data streams, sometimes with limited available computational resources. This is an important task that is rooted in many real-world applications, including and not limited to cybersecurity, medicine and astrophysics. While fast and efficient online algorithms have been recently introduced, these rely on parametric assumptions which are often violated in practical applications. Motivated by data streams from the telecommunications sector, we build a flexible nonparametric approach to detect a change in the distribution of a sequence. Our procedure, NP-FOCuS, builds a sequential likelihood ratio test for a change in a set of points of the empirical cumulative density function of our data. This is achieved by keeping track of the number of observations above or below those points. Thanks to functional pruning ideas, NP-FOCuS has a computational cost that is log-linear in the number of observations and is suitable for high-frequency data streams. In terms of detection power, NP-FOCuS is seen to outperform current nonparametric online changepoint techniques in a variety of settings. We demonstrate the utility of the procedure on both simulated and real data.