Ultra High Dimensional Change Point Detection

TL;DR

This paper introduces a novel two-stage method for detecting change points in ultra high-dimensional data, effectively handling spurious correlations and ensuring asymptotic consistency with promising numerical results.

Contribution

It proposes a dynamic correlation screening method combined with group variable selection for change point detection in ultra high dimensions, addressing spurious correlations.

Findings

Method achieves asymptotic consistency.

Numerical results show promising performance.

Handles spurious correlations effectively.

Abstract

Structural breaks have been commonly seen in applications. Specifically for detection of change points in time, research gap still remains on the setting in ultra high dimension, where the covariates may bear spurious correlations. In this paper, we propose a two-stage approach to detect change points in ultra high dimension, by firstly proposing the dynamic titled current correlation screening method to reduce the input dimension, and then detecting possible change points in the framework of group variable selection. Not only the spurious correlation between ultra-high dimensional covariates is taken into consideration in variable screening, but non-convex penalties are studied in change point detection in the ultra high dimension. Asymptotic properties are derived to guarantee the asymptotic consistency of the selection procedure, and the numerical investigations show the promising performance of the proposed approach.