Tight Lower Bounds for Differentially Private Selection

TL;DR

This paper establishes tight lower bounds on the sample complexity for differentially private selection tasks, showing that existing methods are essentially optimal and extending fingerprinting techniques to sparse settings.

Contribution

It introduces a novel extension of the fingerprinting method to derive tight lower bounds for private selection problems involving sparse query sets.

Findings

Lower bound of n = Ω(√k log d) samples needed for private selection.

Existing algorithms are near-optimal given the lower bounds.

Extension of fingerprinting method to sparse query scenarios.

Abstract

A pervasive task in the differential privacy literature is to select the $k$ items of "highest quality" out of a set of $d$ items, where the quality of each item depends on a sensitive dataset that must be protected. Variants of this task arise naturally in fundamental problems like feature selection and hypothesis testing, and also as subroutines for many sophisticated differentially private algorithms. The standard approaches to these tasks---repeated use of the exponential mechanism or the sparse vector technique---approximately solve this problem given a dataset of $n = O(\sqrt{k}\log d)$ samples. We provide a tight lower bound for some very simple variants of the private selection problem. Our lower bound shows that a sample of size $n = \Omega(\sqrt{k} \log d)$ is required even to achieve a very minimal accuracy guarantee. Our results are based on an extension of the fingerprinting method to sparse selection problems. Previously, the fingerprinting method has been used to provide tight lower bounds for answering an entire set of $d$ queries, but often only some much smaller set of $k$ queries are relevant. Our extension allows us to prove lower bounds that depend on both the number of relevant queries and the total number of queries.