Non-parametric Quickest Change Detection for Large Scale Random Matrices

TL;DR

This paper introduces a non-parametric method for detecting distribution changes in large-scale random matrices, leveraging k-nearest neighbor correlation, and proves its asymptotic optimality in high-dimensional settings.

Contribution

It proposes a novel non-parametric stopping rule based on k-nearest neighbor correlation for quickest change detection in large matrices, with theoretical optimality guarantees.

Findings

The proposed method is asymptotically optimal in large p regimes.

It effectively detects changes with unknown distributions.

The approach is applicable to high-dimensional data with sparse dispersion matrices.

Abstract

The problem of quickest detection of a change in the distribution of a $n\times p$ random matrix based on a sequence of observations having a single unknown change point is considered. The forms of the pre- and post-change distributions of the rows of the matrices are assumed to belong to the family of elliptically contoured densities with sparse dispersion matrices but are otherwise unknown. We propose a non-parametric stopping rule that is based on a novel summary statistic related to k-nearest neighbor correlation between columns of each observed random matrix. In the large scale regime of $p\rightarrow \infty$ and $n$ fixed we show that, among all functions of the proposed summary statistic, the proposed stopping rule is asymptotically optimal under a minimax quickest change detection (QCD) model.