Beating Randomized Response on Incoherent Matrices

TL;DR

This paper presents a new differentially private low rank approximation algorithm that significantly outperforms randomized response methods for incoherent matrices, offering improved accuracy and efficiency.

Contribution

The authors introduce a novel algorithm that surpasses randomized response in accuracy for low coherence matrices while maintaining computational efficiency.

Findings

Achieves better accuracy than randomized response under low coherence assumptions.

Computes a constant rank approximation in O(mn) time, matching the complexity of noise generation.

Provides the first significant improvement over naive randomized response methods for this task.

Abstract

Computing accurate low rank approximations of large matrices is a fundamental data mining task. In many applications however the matrix contains sensitive information about individuals. In such case we would like to release a low rank approximation that satisfies a strong privacy guarantee such as differential privacy. Unfortunately, to date the best known algorithm for this task that satisfies differential privacy is based on naive input perturbation or randomized response: Each entry of the matrix is perturbed independently by a sufficiently large random noise variable, a low rank approximation is then computed on the resulting matrix. We give (the first) significant improvements in accuracy over randomized response under the natural and necessary assumption that the matrix has low coherence. Our algorithm is also very efficient and finds a constant rank approximation of an m x n matrix in time O(mn). Note that even generating the noise matrix required for randomized response already requires time O(mn).