Minimax Rate-optimal Estimation of High-dimensional Covariance Matrices with Incomplete Data

TL;DR

This paper develops minimax rate-optimal estimators for high-dimensional covariance matrices with missing data, providing theoretical guarantees and demonstrating their effectiveness through simulations and real data applications.

Contribution

It introduces new estimators for high-dimensional covariance matrices with missing data and proves their minimax optimality under spectral norm loss.

Findings

Estimators achieve minimax convergence rates.

Proposed methods perform well in simulations.

Applications to ovarian cancer data illustrate practical utility.

Abstract

Missing data occur frequently in a wide range of applications. In this paper, we consider estimation of high-dimensional covariance matrices in the presence of missing observations under a general missing completely at random model in the sense that the missingness is not dependent on the values of the data. Based on incomplete data, estimators for bandable and sparse covariance matrices are proposed and their theoretical and numerical properties are investigated. Minimax rates of convergence are established under the spectral norm loss and the proposed estimators are shown to be rate-optimal under mild regularity conditions. Simulation studies demonstrate that the estimators perform well numerically. The methods are also illustrated through an application to data from four ovarian cancer studies. The key technical tools developed in this paper are of independent interest and potentially useful for a range of related problems in high-dimensional statistical inference with missing data.