A Private Approximation of the 2nd-Moment Matrix of Any Subsamplable Input

TL;DR

This paper introduces a differentially private algorithm for accurately estimating the second moment matrix of data, even with outliers, under subsampling assumptions, balancing privacy and utility effectively.

Contribution

It presents a novel recursive framework for private second moment estimation that works under subsamplability assumptions and handles outliers.

Findings

Achieves strong privacy-utility trade-offs for worst-case inputs.

Handles outliers effectively in second moment matrix estimation.

Provides theoretical guarantees under subsampling assumptions.

Abstract

We study the problem of differentially private second moment estimation and present a new algorithm that achieve strong privacy-utility trade-offs even for worst-case inputs under subsamplability assumptions on the data. We call an input $(m,\alpha,\beta)$-subsamplable if a random subsample of size $m$ (or larger) preserves w.p $\geq 1-\beta$ the spectral structure of the original second moment matrix up to a multiplicative factor of $1\pm \alpha$. Building upon subsamplability, we give a recursive algorithmic framework similar to Kamath et al 2019, that abides zero-Concentrated Differential Privacy (zCDP) while preserving w.h.p. the accuracy of the second moment estimation upto an arbitrary factor of $(1\pm\gamma)$. We then show how to apply our algorithm to approximate the second moment matrix of a distribution $\mathcal{D}$, even when a noticeable fraction of the input are outliers.