Optimal Differentially Private PCA and Estimation for Spiked Covariance Matrices

TL;DR

This paper develops optimal differentially private methods for estimating covariance matrices and principal components in the spiked covariance model, achieving minimax rates and broad applicability including high-dimensional and diverging rank scenarios.

Contribution

It introduces the first minimax optimal differentially private PCA and covariance estimators for the spiked covariance model, handling diverging rank and small sample sizes.

Findings

Proposed computationally efficient private estimators are minimax optimal for sub-Gaussian data.

Derived sensitivity bounds for eigenvalues and eigenvectors under the spiked model.

Validated methods through simulations and real data experiments.

Abstract

Estimating a covariance matrix and its associated principal components is a fundamental problem in contemporary statistics. While optimal estimation procedures have been developed with well-understood properties, the increasing demand for privacy preservation introduces new complexities to this classical problem. In this paper, we study optimal differentially private Principal Component Analysis (PCA) and covariance estimation within the spiked covariance model. We precisely characterize the sensitivity of eigenvalues and eigenvectors under this model and establish the minimax rates of convergence for estimating both the principal components and covariance matrix. These rates hold up to logarithmic factors and encompass general Schatten norms, including spectral norm, Frobenius norm, and nuclear norm as special cases. We propose computationally efficient differentially private estimators and prove their minimax optimality for sub-Gaussian distributions, up to logarithmic factors. Additionally, matching minimax lower bounds are established. Notably, compared to the existing literature, our results accommodate a diverging rank, a broader range of signal strengths, and remain valid even when the sample size is much smaller than the dimension, provided the signal strength is sufficiently strong. Both simulation studies and real data experiments demonstrate the merits of our method.