Narrowest-Over-Threshold Detection of Multiple Change-points and Change-point-like Features

TL;DR

This paper introduces the Narrowest-Over-Threshold (NOT) method for flexible, nonparametric detection of multiple change-points and features in data, with theoretical guarantees and efficient implementation.

Contribution

The paper presents a novel, adaptable change-point detection technique called NOT, capable of handling various feature types with proven consistency and near-optimality, and provides an automatic threshold selection method.

Findings

NOT achieves high accuracy in detecting change-points.

The method is computationally efficient, running in near-linear time.

Performance matches or exceeds existing state-of-the-art methods.

Abstract

We propose a new, generic and flexible methodology for nonparametric function estimation, in which we first estimate the number and locations of any features that may be present in the function, and then estimate the function parametrically between each pair of neighbouring detected features. Examples of features handled by our methodology include change-points in the piecewise-constant signal model, kinks in the piecewise-linear signal model, and other similar irregularities, which we also refer to as generalised change-points. Our methodology works with only minor modifications across a range of generalised change-point scenarios, and we achieve such a high degree of generality by proposing and using a new multiple generalised change-point detection device, termed Narrowest-Over-Threshold (NOT). The key ingredient of NOT is its focus on the smallest local sections of the data on which the existence of a feature is suspected. Crucially, this adaptive localisation technique prevents NOT from considering subsamples containing two or more features, a key factor that ensures the general applicability of NOT. For selected scenarios, we show the consistency and near-optimality of NOT in detecting the number and locations of generalised change-points. Furthermore, we propose to select NOT's threshold (automatically) via the strengthened Schwarz Information Criterion (sSIC) and give theoretical justifications. The NOT estimators are easy to implement and rapid to compute: the entire threshold-indexed solution path can be computed in close-to-linear time. Importantly, the NOT approach is easy to extend by the user to tailor to their own needs. There is no single competitor, but we show that the performance of NOT matches or surpasses the state of the art in the scenarios tested. Our methodology is implemented in the R package \textbf{not}.