Private Multiplicative Weights Beyond Linear Queries

TL;DR

This paper extends private multiplicative weights techniques to accurately solve exponentially many convex minimization problems on sensitive data while preserving differential privacy.

Contribution

It introduces a method for differentially private solutions to multiple convex minimizations, expanding beyond linear queries.

Findings

Achieves accurate private solutions for many convex problems

Extends private multiplicative weights to convex optimization

Supports exponentially many queries with privacy guarantees

Abstract

A wide variety of fundamental data analyses in machine learning, such as linear and logistic regression, require minimizing a convex function defined by the data. Since the data may contain sensitive information about individuals, and these analyses can leak that sensitive information, it is important to be able to solve convex minimization in a privacy-preserving way. A series of recent results show how to accurately solve a single convex minimization problem in a differentially private manner. However, the same data is often analyzed repeatedly, and little is known about solving multiple convex minimization problems with differential privacy. For simpler data analyses, such as linear queries, there are remarkable differentially private algorithms such as the private multiplicative weights mechanism (Hardt and Rothblum, FOCS 2010) that accurately answer exponentially many distinct queries. In this work, we extend these results to the case of convex minimization and show how to give accurate and differentially private solutions to *exponentially many* convex minimization problems on a sensitive dataset.