The Bernstein Mechanism: Function Release under Differential Privacy

TL;DR

This paper introduces the Bernstein Mechanism, a new method for releasing functions under differential privacy, using polynomial approximation and perturbation, applicable to various algorithms including SVMs and density estimators.

Contribution

It develops a general functional mechanism leveraging Bernstein operators for differentially private function release, applicable to black-box functions and a broad range of learning algorithms.

Findings

Achieves fast utility rates under weak regularity conditions

Provides a lower bound on utility for any differentially private mechanism

Demonstrates competitive results on kernel density estimation and SVMs

Abstract

We address the problem of general function release under differential privacy, by developing a functional mechanism that applies under the weak assumptions of oracle access to target function evaluation and sensitivity. These conditions permit treatment of functions described explicitly or implicitly as algorithmic black boxes. We achieve this result by leveraging the iterated Bernstein operator for polynomial approximation of the target function, and polynomial coefficient perturbation. Under weak regularity conditions, we establish fast rates on utility measured by high-probability uniform approximation. We provide a lower bound on the utility achievable for any functional mechanism that is $\varepsilon$-differentially private. The generality of our mechanism is demonstrated by the analysis of a number of example learners, including naive Bayes, non-parametric estimators and regularized empirical risk minimization. Competitive rates are demonstrated for kernel density estimation; and $\varepsilon$-differential privacy is achieved for a broader class of support vector machines than known previously.