Polynomial Time and Private Learning of Unbounded Gaussian Mixture Models

TL;DR

This paper introduces a polynomial-time, differentially private algorithm for learning unbounded Gaussian Mixture Models, reducing the problem to non-private algorithms and establishing new bounds without parameter restrictions.

Contribution

It develops a blackbox reduction technique for private learning of GMMs, enabling the first polynomial-time, unbounded parameter private learning algorithm with sample complexity bounds.

Findings

First polynomial-time private GMM learning algorithm without boundedness assumptions

New sample complexity upper bounds for private GMM estimation

Proved a tight lower bound on total variation distance for high-dimensional Gaussians

Abstract

We study the problem of privately estimating the parameters of $d$-dimensional Gaussian Mixture Models (GMMs) with $k$ components. For this, we develop a technique to reduce the problem to its non-private counterpart. This allows us to privatize existing non-private algorithms in a blackbox manner, while incurring only a small overhead in the sample complexity and running time. As the main application of our framework, we develop an $(\varepsilon, \delta)$-differentially private algorithm to learn GMMs using the non-private algorithm of Moitra and Valiant [MV10] as a blackbox. Consequently, this gives the first sample complexity upper bound and first polynomial time algorithm for privately learning GMMs without any boundedness assumptions on the parameters. As part of our analysis, we prove a tight (up to a constant factor) lower bound on the total variation distance of high-dimensional Gaussians which can be of independent interest.