Statistically and Computationally Efficient Change Point Localization in Regression Settings

TL;DR

This paper introduces VPWBS, a novel algorithm for efficiently detecting multiple change points in high-dimensional regression, achieving near-parametric localization rates and outperforming existing methods in accuracy and robustness.

Contribution

The paper proposes VPWBS, a projection-based method that transforms high-dimensional change-point detection into a simpler mean change detection problem, with improved localization rates.

Findings

VPWBS achieves an $O_p(1/n)$ localization rate, better than the previous $O_p(1/\sqrt{n})$.

VPWBS outperforms state-of-the-art algorithms in numerical experiments, especially with small coefficient changes.

The method is robust and computationally efficient for high-dimensional regression change-point detection.

Abstract

Detecting when the underlying distribution changes for the observed time series is a fundamental problem arising in a broad spectrum of applications. In this paper, we study multiple change-point localization in the high-dimensional regression setting, which is particularly challenging as no direct observations of the parameter of interest is available. Specifically, we assume we observe $\{ x_t, y_t\}_{t=1}^n$ where $ \{ x_t\}_{t=1}^n $ are $p$-dimensional covariates, $\{y_t\}_{t=1}^n$ are the univariate responses satisfying $\mathbb{E}(y_t) = x_t^\top \beta_t^* \text{ for } 1\le t \le n $ and $\{\beta_t^*\}_{t=1}^n $ are the unobserved regression coefficients that change over time in a piecewise constant manner. We propose a novel projection-based algorithm, Variance Projected Wild Binary Segmentation~(VPWBS), which transforms the original (difficult) problem of change-point detection in $p$-dimensional regression to a simpler problem of change-point detection in mean of a one-dimensional time series. VPWBS is shown to achieve sharp localization rate $O_p(1/n)$ up to a log factor, a significant improvement from the best rate $O_p(1/\sqrt{n})$ known in the existing literature for multiple change-point localization in high-dimensional regression. Extensive numerical experiments are conducted to demonstrate the robust and favorable performance of VPWBS over two state-of-the-art algorithms, especially when the size of change in the regression coefficients $\{\beta_t^*\}_{t=1}^n $ is small.