Robust and Differentially Private PCA for non-Gaussian data

TL;DR

This paper introduces a differentially private PCA method that is robust to heavy-tailed and contaminated data, overcoming limitations of previous approaches by leveraging elliptical distribution properties and bounded transformations.

Contribution

The paper presents a novel differentially private PCA technique applicable to heavy-tailed and contaminated data, with theoretical guarantees and improved empirical performance.

Findings

Outperforms existing methods in non-Gaussian data scenarios

Provides robustness against data contamination

Achieves effective private subspace recovery

Abstract

Recent advances have sparked significant interest in the development of privacy-preserving Principal Component Analysis (PCA). However, many existing approaches rely on restrictive assumptions, such as assuming sub-Gaussian data or being vulnerable to data contamination. Additionally, some methods are computationally expensive or depend on unknown model parameters that must be estimated, limiting their accessibility for data analysts seeking privacy-preserving PCA. In this paper, we propose a differentially private PCA method applicable to heavy-tailed and potentially contaminated data. Our approach leverages the property that the covariance matrix of properly rescaled data preserves eigenvectors and their order under elliptical distributions, which include Gaussian and heavy-tailed distributions. By applying a bounded transformation, we enable straightforward computation of principal components in a differentially private manner. Additionally, boundedness guarantees robustness against data contamination. We conduct both theoretical analysis and empirical evaluations of the proposed method, focusing on its ability to recover the subspace spanned by the leading principal components. Extensive numerical experiments demonstrate that our method consistently outperforms existing approaches in terms of statistical utility, particularly in non-Gaussian or contaminated data settings.