A Framework for Private Matrix Analysis

TL;DR

This paper introduces efficient differentially private algorithms for matrix analysis tasks in the sliding window model, including spectral approximation, PCA, and linear regression, with novel results for sparse and non-negative PCA.

Contribution

It provides the first efficient private algorithms for sparse and non-negative PCA in streaming data, along with space lower bounds for low-rank approximation.

Findings

Efficient $o(W)$ space algorithms for spectral approximation, PCA, and linear regression.

First known private algorithms for sparse and non-negative PCA in streaming settings.

A space lower bound for low-rank approximation under differential privacy.

Abstract

We study private matrix analysis in the sliding window model where only the last $W$ updates to matrices are considered useful for analysis. We give first efficient $o(W)$ space differentially private algorithms for spectral approximation, principal component analysis, and linear regression. We also initiate and show efficient differentially private algorithms for two important variants of principal component analysis: sparse principal component analysis and non-negative principal component analysis. Prior to our work, no such result was known for sparse and non-negative differentially private principal component analysis even in the static data setting. These algorithms are obtained by identifying sufficient conditions on positive semidefinite matrices formed from streamed matrices. We also show a lower bound on space required to compute low-rank approximation even if the algorithm gives multiplicative approximation and incurs additive error. This follows via reduction to a certain communication complexity problem.