The Influence of Dropout on Membership Inference in Differentially Private Models

TL;DR

This paper investigates how dropout, a method for uncertainty quantification, affects the privacy of models trained with differential privacy, revealing that large dropout can slightly increase membership inference risks.

Contribution

It provides the first analysis of the interaction between dropout and differential privacy in the context of membership inference attacks.

Findings

Large dropout slightly increases membership inference risk.

Differential privacy reduces vulnerability but does not eliminate risks.

Dropout's impact varies with privacy settings.

Abstract

Differentially private models seek to protect the privacy of data the model is trained on, making it an important component of model security and privacy. At the same time, data scientists and machine learning engineers seek to use uncertainty quantification methods to ensure models are as useful and actionable as possible. We explore the tension between uncertainty quantification via dropout and privacy by conducting membership inference attacks against models with and without differential privacy. We find that models with large dropout slightly increases a model's risk to succumbing to membership inference attacks in all cases including in differentially private models.