A Learning Theory Approach to Non-Interactive Database Privacy

TL;DR

This paper explores the limits of non-interactive data privacy, proposing mechanisms for synthetic data release with utility guarantees based on learning theory, and introduces a new privacy notion called distributional privacy.

Contribution

It introduces a learning-theoretic framework for non-interactive privacy mechanisms, including new algorithms and the concept of distributional privacy, strengthening privacy guarantees.

Findings

Synthetic data release is feasible for large query classes with error depending on VC-dimension.

Releasing synthetic data for simple classes over continuous domains is impossible.

A polynomial-time algorithm for halfspace queries with relaxed utility guarantees is provided.

Abstract

In this paper we demonstrate that, ignoring computational constraints, it is possible to privately release synthetic databases that are useful for large classes of queries -- much larger in size than the database itself. Specifically, we give a mechanism that privately releases synthetic data for a class of queries over a discrete domain with error that grows as a function of the size of the smallest net approximately representing the answers to that class of queries. We show that this in particular implies a mechanism for counting queries that gives error guarantees that grow only with the VC-dimension of the class of queries, which itself grows only logarithmically with the size of the query class. We also show that it is not possible to privately release even simple classes of queries (such as intervals and their generalizations) over continuous domains. Despite this, we give a privacy-preserving polynomial time algorithm that releases information useful for all halfspace queries, given a slight relaxation of the utility guarantee. This algorithm does not release synthetic data, but instead another data structure capable of representing an answer for each query. We also give an efficient algorithm for releasing synthetic data for the class of interval queries and axis-aligned rectangles of constant dimension. Finally, inspired by learning theory, we introduce a new notion of data privacy, which we call distributional privacy, and show that it is strictly stronger than the prevailing privacy notion, differential privacy.