Private Prediction Sets

TL;DR

This paper introduces a method combining conformal prediction with differential privacy to produce reliable, privacy-preserving prediction sets for machine learning models, validated on large-scale datasets.

Contribution

It develops a novel approach that applies differential privacy to conformal prediction, enabling private uncertainty quantification for pre-trained models.

Findings

Guarantees 90% coverage with privacy preservation.

Effective on large-scale computer vision datasets.

Provides a practical framework for privacy-aware uncertainty quantification.

Abstract

In real-world settings involving consequential decision-making, the deployment of machine learning systems generally requires both reliable uncertainty quantification and protection of individuals' privacy. We present a framework that treats these two desiderata jointly. Our framework is based on conformal prediction, a methodology that augments predictive models to return prediction sets that provide uncertainty quantification -- they provably cover the true response with a user-specified probability, such as 90%. One might hope that when used with privately-trained models, conformal prediction would yield privacy guarantees for the resulting prediction sets; unfortunately, this is not the case. To remedy this key problem, we develop a method that takes any pre-trained predictive model and outputs differentially private prediction sets. Our method follows the general approach of split conformal prediction; we use holdout data to calibrate the size of the prediction sets but preserve privacy by using a privatized quantile subroutine. This subroutine compensates for the noise introduced to preserve privacy in order to guarantee correct coverage. We evaluate the method on large-scale computer vision datasets.