Adaptive Power Iteration Method for Differentially Private PCA

TL;DR

This paper introduces an adaptive power iteration algorithm for differentially private PCA that leverages matrix coherence to improve performance beyond worst-case scenarios.

Contribution

It presents a novel adaptive filtering technique for private power iteration that exploits low coherence in matrices, enhancing privacy-utility trade-offs.

Findings

Achieves better utility for matrices with low coherence.

Introduces a new filtering technique for adaptive privacy.

Extends beyond worst-case guarantees in private PCA.

Abstract

We study $\left(\epsilon,\delta\right)$-differentially private algorithms for the problem of approximately computing the top singular vector of a matrix $A\in\mathbb{R}^{n\times d}$ where each row of $A$ is a data point in $\mathbb{R}^{d}$. Following Dwork-Talwar-Thakurta-Zhang (STOC 2014), we consider the privacy model where neighboring inputs differ by one single row. We give a novel algorithm that achieves beyond-worst-case guarantees for input matrices with low coherence, which is a structural property of matrices in many applications, including but not limited to i.i.d. data. Our algorithm contributes to the extensive literature on private power iteration methods, where we introduce a new filtering technique which adapts to this coherence parameter. Our work departs from and complements the work by Hardt-Roth (STOC 2013) which achieves beyond-worst-case guarantees for the more restrictive privacy model where neighboring inputs differ in one single entry by at most 1.