Data Distribution Shifts in (Industrial) Federated Learning as a Privacy Issue

TL;DR

This paper investigates privacy risks in industrial federated learning, demonstrating that subtle data distribution shifts can be detected by adversaries, potentially revealing sensitive information about competitors' production changes.

Contribution

It introduces a method for detecting subtle distribution shifts in federated learning models, highlighting privacy vulnerabilities specific to industrial collaborations.

Findings

Adversaries can detect distribution shifts before they are evident in evaluation metrics.

Subtle data changes can be inferred from model internal states.

Industrial federated learning poses unique privacy risks not present in cross-device settings.

Abstract

We consider industrial federated learning, a collaboration between a small number of powerful, potentially competing industrial players, mediated by a third party aspiring to improve the service it provides to its customers. We argue that this configuration harbours covert privacy risks that do not arise in e.g. cross-device settings. Companies are very protective of their intellectual property and production processes. Information about changes to their production and the timing of which is to be kept private. We study a scenario in which one of the collaborators infers changes to their competitors' production by detecting potentially subtle temporal data distribution shifts. In this framing, a data distribution shift is always problematic, even if it has no negative effect on training convergence. Thus, our goal is to find means that allow the detection of distributional shifts better than customary evaluation metrics. Based on the assumption that even minor shifts translate into the collaboratively learned machine learning model, the attacker tracks the shared models' internal state with a selection of metrics from literature in order to pick up on relevant changes. In an empirical study on benchmark datasets, we show an honest-but-curious attacker to be capable of detecting subtle distributional shifts on other clients, in some cases long before they become obvious in evaluation.