Differentially Private Optimizers Can Learn Adversarially Robust Models

TL;DR

This paper demonstrates that differentially private training can produce models that are both accurate and adversarially robust, sometimes outperforming non-private models, challenging prior assumptions about privacy-robustness tradeoffs.

Contribution

First theoretical and empirical analysis showing DP models can be robust and accurate, highlighting key factors influencing the privacy-robustness-accuracy tradeoff.

Findings

DP models can be more robust than non-private models.

Proper hyper-parameters and pre-training improve robustness.

DP models are Pareto optimal on the accuracy-robustness tradeoff.

Abstract

Machine learning models have shone in a variety of domains and attracted increasing attention from both the security and the privacy communities. One important yet worrying question is: Will training models under the differential privacy (DP) constraint have an unfavorable impact on their adversarial robustness? While previous works have postulated that privacy comes at the cost of worse robustness, we give the first theoretical analysis to show that DP models can indeed be robust and accurate, even sometimes more robust than their naturally-trained non-private counterparts. We observe three key factors that influence the privacy-robustness-accuracy tradeoff: (1) hyper-parameters for DP optimizers are critical; (2) pre-training on public data significantly mitigates the accuracy and robustness drop; (3) choice of DP optimizers makes a difference. With these factors set properly, we achieve 90\% natural accuracy, 72\% robust accuracy ($+9\%$ than the non-private model) under $l_2(0.5)$ attack, and 69\% robust accuracy ($+16\%$ than the non-private model) with pre-trained SimCLRv2 model under $l_\infty(4/255)$ attack on CIFAR10 with $\epsilon=2$. In fact, we show both theoretically and empirically that DP models are Pareto optimal on the accuracy-robustness tradeoff. Empirically, the robustness of DP models is consistently observed across various datasets and models. We believe our encouraging results are a significant step towards training models that are private as well as robust.