Gradient-Leaks: Understanding and Controlling Deanonymization in Federated Learning

TL;DR

This paper investigates how model updates in federated learning can reveal user identities, demonstrating deanonymization risks and proposing data-augmentation defenses that balance privacy and utility.

Contribution

It uncovers the encoding of user-specific information in model updates and introduces data-augmentation strategies to mitigate deanonymization risks.

Findings

Model updates encode subtle user-specific variations.

Adversaries can deanonymize devices with limited auxiliary data.

Data-augmentation strategies effectively reduce deanonymization risk.

Abstract

Federated Learning (FL) systems are gaining popularity as a solution to training Machine Learning (ML) models from large-scale user data collected on personal devices (e.g., smartphones) without their raw data leaving the device. At the core of FL is a network of anonymous user devices sharing training information (model parameter updates) computed locally on personal data. However, the type and degree to which user-specific information is encoded in the model updates is poorly understood. In this paper, we identify model updates encode subtle variations in which users capture and generate data. The variations provide a strong statistical signal, allowing an adversary to effectively deanonymize participating devices using a limited set of auxiliary data. We analyze resulting deanonymization attacks on diverse tasks on real-world (anonymized) user-generated data across a range of closed- and open-world scenarios. We study various strategies to mitigate the risks of deanonymization. As random perturbation methods do not offer convincing operating points, we propose data-augmentation strategies which introduces adversarial biases in device data and thereby, offer substantial protection against deanonymization threats with little effect on utility.