DP-SGD vs PATE: Which Has Less Disparate Impact on Model Accuracy?

TL;DR

This paper compares DP-SGD and PATE mechanisms for differentially private deep learning, focusing on their fairness impacts, and finds PATE generally causes less disparate impact on model accuracy across subgroups.

Contribution

The study provides a comparative analysis of DP-SGD and PATE in terms of fairness, highlighting PATE's relatively lower disparate impact and suggesting directions for improved fairness-privacy trade-offs.

Findings

PATE has less severe disparate impact than DP-SGD.

Both mechanisms exhibit some level of fairness disparity.

Insights suggest potential for better fairness-privacy balance.

Abstract

Recent advances in differentially private deep learning have demonstrated that application of differential privacy, specifically the DP-SGD algorithm, has a disparate impact on different sub-groups in the population, which leads to a significantly high drop-in model utility for sub-populations that are under-represented (minorities), compared to well-represented ones. In this work, we aim to compare PATE, another mechanism for training deep learning models using differential privacy, with DP-SGD in terms of fairness. We show that PATE does have a disparate impact too, however, it is much less severe than DP-SGD. We draw insights from this observation on what might be promising directions in achieving better fairness-privacy trade-offs.