On Low-Space Differentially Private Low-rank Factorization in the Spectral Norm

TL;DR

This paper introduces two efficient, sub-linear space algorithms for differentially private low-rank matrix factorization in the spectral norm within the turnstile update model, advancing privacy and computational efficiency in spectral analysis of sensitive data.

Contribution

It presents the first non-private and private algorithms for spectral norm low-rank factorization in the turnstile model, with stronger privacy guarantees and improved efficiency.

Findings

Two algorithms with sub-linear space complexity

Stronger privacy guarantees than prior work

First algorithms for spectral norm low-rank factorization in turnstile model

Abstract

Low-rank factorization is used in many areas of computer science where one performs spectral analysis on large sensitive data stored in the form of matrices. In this paper, we study differentially private low-rank factorization of a matrix with respect to the spectral norm in the turnstile update model. In this problem, given an input matrix $\mathbf{A} \in \mathbb{R}^{m \times n}$ updated in the turnstile manner and a target rank $k$, the goal is to find two rank-$k$ orthogonal matrices $\mathbf{U}_k \in \mathbb{R}^{m \times k}$ and $\mathbf{V}_k \in \mathbb{R}^{n \times k}$, and one positive semidefinite diagonal matrix $\textbf{\Sigma}_k \in \mathbb{R}^{k \times k}$ such that $\mathbf{A} \approx \mathbf{U}_k \textbf{\Sigma}_k \mathbf{V}_k^\mathsf{T}$ with respect to the spectral norm. Our main contributions are two computationally efficient and sub-linear space algorithms for computing a differentially private low-rank factorization. We consider two levels of privacy. In the first level of privacy, we consider two matrices neighboring if their difference has a Frobenius norm at most $1$. In the second level of privacy, we consider two matrices as neighboring if their difference can be represented as an outer product of two unit vectors. Both these privacy levels are stronger than those studied in the earlier papers such as Dwork {\it et al.} (STOC 2014), Hardt and Roth (STOC 2013), and Hardt and Price (NIPS 2014). As a corollary to our results, we get non-private algorithms that compute low-rank factorization in the turnstile update model with respect to the spectral norm. We note that, prior to this work, no algorithm that outputs low-rank factorization with respect to the spectral norm in the turnstile update model was known; i.e., our algorithm gives the first non-private low-rank factorization with respect to the spectral norm in the turnstile update mode.