Mixed Differential Privacy in Computer Vision

TL;DR

AdaMix is an adaptive differentially private algorithm that improves the privacy-accuracy trade-off in training deep neural classifiers on image data by combining public and private datasets with strong theoretical guarantees.

Contribution

It introduces AdaMix, a novel method that integrates few-shot and zero-shot learning techniques with differential privacy for vision tasks, addressing privacy-accuracy trade-offs.

Findings

Reduces error increase from 167-311% to 68-92% across datasets.

Effectively combines public and private data for improved privacy-utility balance.

Provides strong theoretical privacy guarantees and convergence analysis.

Abstract

We introduce AdaMix, an adaptive differentially private algorithm for training deep neural network classifiers using both private and public image data. While pre-training language models on large public datasets has enabled strong differential privacy (DP) guarantees with minor loss of accuracy, a similar practice yields punishing trade-offs in vision tasks. A few-shot or even zero-shot learning baseline that ignores private data can outperform fine-tuning on a large private dataset. AdaMix incorporates few-shot training, or cross-modal zero-shot learning, on public data prior to private fine-tuning, to improve the trade-off. AdaMix reduces the error increase from the non-private upper bound from the 167-311\% of the baseline, on average across 6 datasets, to 68-92\% depending on the desired privacy level selected by the user. AdaMix tackles the trade-off arising in visual classification, whereby the most privacy sensitive data, corresponding to isolated points in representation space, are also critical for high classification accuracy. In addition, AdaMix comes with strong theoretical privacy guarantees and convergence analysis.