Exploiting Unintended Feature Leakage in Collaborative Learning

TL;DR

This paper reveals that collaborative and federated learning updates can unintentionally leak sensitive training data information, and introduces attacks to exploit this leakage, highlighting privacy risks and potential defenses.

Contribution

It demonstrates novel inference attacks that extract private data details from model updates in collaborative learning, exposing privacy vulnerabilities.

Findings

Adversaries can perform membership inference to identify specific data points.

Attacks can infer properties of training data independent of the model's intended features.

The paper evaluates attack effectiveness across various datasets and discusses defense strategies.

Abstract

Collaborative machine learning and related techniques such as federated learning allow multiple participants, each with his own training dataset, to build a joint model by training locally and periodically exchanging model updates. We demonstrate that these updates leak unintended information about participants' training data and develop passive and active inference attacks to exploit this leakage. First, we show that an adversarial participant can infer the presence of exact data points -- for example, specific locations -- in others' training data (i.e., membership inference). Then, we show how this adversary can infer properties that hold only for a subset of the training data and are independent of the properties that the joint model aims to capture. For example, he can infer when a specific person first appears in the photos used to train a binary gender classifier. We evaluate our attacks on a variety of tasks, datasets, and learning configurations, analyze their limitations, and discuss possible defenses.