Analyzing Inference Privacy Risks Through Gradients in Machine Learning

TL;DR

This paper systematically analyzes privacy risks in distributed machine learning by examining how gradients can leak sensitive information, evaluating defenses, and proposing an auditing method to improve privacy estimation.

Contribution

It introduces a unified game-based framework for analyzing gradient-based privacy risks and evaluates multiple defenses, providing an information-theoretic perspective and an auditing method.

Findings

Data aggregation alone is ineffective against inference attacks.

Several defenses reduce privacy leakage but vary in effectiveness.

The auditing method improves worst-case privacy estimation.

Abstract

In distributed learning settings, models are iteratively updated with shared gradients computed from potentially sensitive user data. While previous work has studied various privacy risks of sharing gradients, our paper aims to provide a systematic approach to analyze private information leakage from gradients. We present a unified game-based framework that encompasses a broad range of attacks including attribute, property, distributional, and user disclosures. We investigate how different uncertainties of the adversary affect their inferential power via extensive experiments on five datasets across various data modalities. Our results demonstrate the inefficacy of solely relying on data aggregation to achieve privacy against inference attacks in distributed learning. We further evaluate five types of defenses, namely, gradient pruning, signed gradient descent, adversarial perturbations, variational information bottleneck, and differential privacy, under both static and adaptive adversary settings. We provide an information-theoretic view for analyzing the effectiveness of these defenses against inference from gradients. Finally, we introduce a method for auditing attribute inference privacy, improving the empirical estimation of worst-case privacy through crafting adversarial canary records.