Learning Robust and Privacy-Preserving Representations via Information Theory

TL;DR

This paper introduces an information-theoretic framework for learning representations that are simultaneously robust to adversarial attacks and preserve privacy, balancing security, privacy, and utility in machine learning models.

Contribution

It proposes a novel framework that unifies robustness and privacy preservation in representation learning, with theoretical insights into their inherent trade-offs.

Findings

Identifies the trade-off between robustness, privacy, and utility.

Provides theoretical guarantees on attribute privacy leakage.

Establishes the inherent limitations in achieving both robustness and privacy.

Abstract

Machine learning models are vulnerable to both security attacks (e.g., adversarial examples) and privacy attacks (e.g., private attribute inference). We take the first step to mitigate both the security and privacy attacks, and maintain task utility as well. Particularly, we propose an information-theoretic framework to achieve the goals through the lens of representation learning, i.e., learning representations that are robust to both adversarial examples and attribute inference adversaries. We also derive novel theoretical results under our framework, e.g., the inherent trade-off between adversarial robustness/utility and attribute privacy, and guaranteed attribute privacy leakage against attribute inference adversaries.