Higher-criticism for sparse multi-stream change-point detection

TL;DR

This paper introduces a higher criticism-based statistical method for detecting sparse change-points across multiple data streams, providing theoretical guarantees and demonstrating superior performance over existing approaches.

Contribution

It develops a novel HC-based procedure for multi-stream change detection, with theoretical analysis and optimality results under a sparse heteroscedastic normal model.

Findings

HC-based method achieves asymptotic optimality in detection delay

Theoretical lower bounds match the HC method's performance

Numerical experiments show improved detection over existing methods

Abstract

We study a statistical procedure based on higher criticism (HC) to address the sparse multi-stream quickest change-point detection problem. Namely, we aim to detect a potential change in the distribution of multiple data streams at some unknown time. If a change occurs, only a few streams are affected, whereas the identity of the affected streams is unknown. The HC-based procedure involves testing for a change point in individual streams and combining multiple tests using higher criticism. Relying on HC thresholding, the procedure also indicates a set of streams suspected to be affected by the change. We provide a theoretical analysis under a sparse heteroscedastic normal change-point model. We establish an information-theoretic detection delay lower bound when individual tests are based on the likelihood ratio or the generalized likelihood ratio statistics and show that the delay of the HC-based method converges in distribution to this bound. In the special case of constant variance, our bound coincides with known results in (Chan, 2017). We demonstrate the effectiveness of the HC-based method compared to other methods in detecting sparse changes through extensive numerical evaluations.