Agnostic Private Density Estimation for GMMs via List Global Stability

TL;DR

This paper introduces the first sample complexity bounds for private density estimation of high-dimensional Gaussian mixture models in the agnostic setting, utilizing a novel agnostic list global stability framework.

Contribution

It develops an agnostic variant of list global stability and constructs a stable learner for GMMs, advancing private density estimation theory.

Findings

First upper bound on sample complexity for private GMM density estimation in agnostic setting

Introduces agnostic list global stability concept for private learning

Constructs a stable learner for high-dimensional GMMs

Abstract

We consider the problem of private density estimation for mixtures of unrestricted high dimensional Gaussians in the agnostic setting. We prove the first upper bound on the sample complexity of this problem. Previously, private learnability of high dimensional GMMs was only known in the realizable setting [Afzali et al., 2024]. To prove our result, we exploit the notion of $\textit{list global stability}$ [Ghazi et al., 2021b,a] that was originally introduced in the context of private supervised learning. We define an agnostic variant of this definition, showing that its existence is sufficient for agnostic private density estimation. We then construct an agnostic list globally stable learner for GMMs.