Convex Banding of the Covariance Matrix

TL;DR

This paper presents a convex optimization-based estimator for high-dimensional covariance matrices that adaptively tapers the sample covariance, achieving optimal theoretical properties and practical effectiveness in classification tasks.

Contribution

It introduces a novel convex banding estimator that is adaptive, minimax rate optimal, and computationally efficient, improving over existing methods in accuracy and speed.

Findings

Estimator is minimax rate adaptive in Frobenius and operator norms.

It correctly recovers the true bandwidth when the covariance is exactly banded.

Empirical results show improved classification performance using the estimator.

Abstract

We introduce a new sparse estimator of the covariance matrix for high-dimensional models in which the variables have a known ordering. Our estimator, which is the solution to a convex optimization problem, is equivalently expressed as an estimator which tapers the sample covariance matrix by a Toeplitz, sparsely-banded, data-adaptive matrix. As a result of this adaptivity, the convex banding estimator enjoys theoretical optimality properties not attained by previous banding or tapered estimators. In particular, our convex banding estimator is minimax rate adaptive in Frobenius and operator norms, up to log factors, over commonly-studied classes of covariance matrices, and over more general classes. Furthermore, it correctly recovers the bandwidth when the true covariance is exactly banded. Our convex formulation admits a simple and efficient algorithm. Empirical studies demonstrate its practical effectiveness and illustrate that our exactly-banded estimator works well even when the true covariance matrix is only close to a banded matrix, confirming our theoretical results. Our method compares favorably with all existing methods, in terms of accuracy and speed. We illustrate the practical merits of the convex banding estimator by showing that it can be used to improve the performance of discriminant analysis for classifying sound recordings.