Assessing Differentially Private Variational Autoencoders under Membership Inference

TL;DR

This paper evaluates the privacy-accuracy trade-offs of differentially private Variational Autoencoders, analyzing reconstruction attacks and privacy parameter settings across various data types and mechanisms, revealing dataset and architecture dependencies.

Contribution

It introduces a comprehensive evaluation of privacy-accuracy trade-offs in differentially private VAEs, including attack assessment and privacy parameter guidance.

Findings

Privacy-accuracy trade-offs vary significantly with dataset and model architecture.

Rarely do VAEs show favorable privacy-accuracy trade-offs.

LDP can outperform CDP in certain scenarios.

Abstract

We present an approach to quantify and compare the privacy-accuracy trade-off for differentially private Variational Autoencoders. Our work complements previous work in two aspects. First, we evaluate the the strong reconstruction MI attack against Variational Autoencoders under differential privacy. Second, we address the data scientist's challenge of setting privacy parameter epsilon, which steers the differential privacy strength and thus also the privacy-accuracy trade-off. In our experimental study we consider image and time series data, and three local and central differential privacy mechanisms. We find that the privacy-accuracy trade-offs strongly depend on the dataset and model architecture. We do rarely observe favorable privacy-accuracy trade-off for Variational Autoencoders, and identify a case where LDP outperforms CDP.