Optimal hypothesis testing for high dimensional covariance matrices

TL;DR

This paper develops a minimax optimal test for high-dimensional covariance matrices, introducing a U-statistic based method that outperforms existing tests across various regimes of the dimension-to-sample-size ratio.

Contribution

It characterizes the boundary of testability using Frobenius norm and proposes a rate-optimal U-statistic based test for high-dimensional covariance hypothesis testing.

Findings

The U-statistic test is rate optimal in the bounded p/n regime.

The U-statistic test uniformly dominates the corrected likelihood ratio test.

The power analysis extends to unbounded p/n regimes.

Abstract

This paper considers testing a covariance matrix $\Sigma$ in the high dimensional setting where the dimension $p$ can be comparable or much larger than the sample size $n$. The problem of testing the hypothesis $H_0:\Sigma=\Sigma_0$ for a given covariance matrix $\Sigma_0$ is studied from a minimax point of view. We first characterize the boundary that separates the testable region from the non-testable region by the Frobenius norm when the ratio between the dimension $p$ over the sample size $n$ is bounded. A test based on a $U$-statistic is introduced and is shown to be rate optimal over this asymptotic regime. Furthermore, it is shown that the power of this test uniformly dominates that of the corrected likelihood ratio test (CLRT) over the entire asymptotic regime under which the CLRT is applicable. The power of the $U$-statistic based test is also analyzed when $p/n$ is unbounded.