Sharp minimax tests for large covariance matrices and adaptation

TL;DR

This paper develops sharp minimax tests for detecting correlations in high-dimensional Gaussian vectors, establishing optimal rates and constructing adaptive procedures for large covariance matrices.

Contribution

It introduces a new U-statistic based test with optimal weights, providing asymptotic and sharp minimax separation rates, and extends results to inverse covariance matrix testing.

Findings

Test statistic is asymptotically Gaussian under null and alternative hypotheses.

Derived sharp minimax separation rates for different smoothness parameters.

Constructed adaptive test procedures that attain near-optimal rates.

Abstract

We consider the detection problem of correlations in a $p$-dimensional Gaussian vector, when we observe $n$ independent, identically distributed random vectors, for $n$ and $p$ large. We assume that the covariance matrix varies in some ellipsoid with parameter $\alpha >1/2$ and total energy bounded by $L>0$. We propose a test procedure based on a U-statistic of order 2 which is weighted in an optimal way. The weights are the solution of an optimization problem, they are constant on each diagonal and non-null only for the $T$ first diagonals, where $T=o(p)$. We show that this test statistic is asymptotically Gaussian distributed under the null hypothesis and also under the alternative hypothesis for matrices close to the detection boundary. We prove upper bounds for the total error probability of our test procedure, for $\alpha>1/2$ and under the assumption $T=o(p)$ which implies that $n=o(p^{ 2 \alpha})$. We illustrate via a numerical study the behavior of our test procedure. Moreover, we prove lower bounds for the maximal type II error and the total error probabilities. Thus we obtain the asymptotic and the sharp asymptotically minimax separation rate $\tilde{\varphi} = (C(\alpha, L) n^2 p )^{- \alpha/(4 \alpha + 1)}$, for $\alpha>3/2$ and for $\alpha >1$ together with the additional assumption $p= o(n^{4 \alpha -1})$, respectively. We deduce rate asymptotic minimax results for testing the inverse of the covariance matrix. We construct an adaptive test procedure with respect to the parameter $\alpha$ and show that it attains the rate $\tilde{\psi}= ( n^2 p / \ln\ln(n \displaystyle\sqrt{p}) )^{- \alpha/(4 \alpha + 1)}$.