Generative Models with Information-Theoretic Protection Against Membership Inference Attacks

TL;DR

This paper introduces an information-theoretic regularization for GANs that enhances privacy against membership inference attacks while maintaining high sample quality and improving downstream classification.

Contribution

It proposes a novel regularization method based on Jensen-Shannon divergence to prevent overfitting and protect privacy in generative models, with low computational overhead.

Findings

Regularization reduces vulnerability to membership inference attacks.

Generated samples retain high quality and improve downstream classification.

Method outperforms non-private and differentially private models in privacy and utility.

Abstract

Deep generative models, such as Generative Adversarial Networks (GANs), synthesize diverse high-fidelity data samples by estimating the underlying distribution of high dimensional data. Despite their success, GANs may disclose private information from the data they are trained on, making them susceptible to adversarial attacks such as membership inference attacks, in which an adversary aims to determine if a record was part of the training set. We propose an information theoretically motivated regularization term that prevents the generative model from overfitting to training data and encourages generalizability. We show that this penalty minimizes the JensenShannon divergence between components of the generator trained on data with different membership, and that it can be implemented at low cost using an additional classifier. Our experiments on image datasets demonstrate that with the proposed regularization, which comes at only a small added computational cost, GANs are able to preserve privacy and generate high-quality samples that achieve better downstream classification performance compared to non-private and differentially private generative models.