Task-Agnostic Privacy-Preserving Representation Learning for Federated Learning Against Attribute Inference Attacks

TL;DR

This paper introduces TAPPFL, a task-agnostic, information-theoretic method for federated learning that effectively protects private attributes against inference attacks while preserving model utility and computational efficiency.

Contribution

The paper proposes TAPPFL, a novel privacy-preserving federated learning approach that does not require prior task knowledge and offers provable privacy guarantees with minimal utility loss.

Findings

TAPPFL effectively reduces attribute inference risks in federated learning.

TAPPFL maintains high model utility comparable to non-private methods.

Experimental results outperform existing privacy defenses in efficiency and privacy protection.

Abstract

Federated learning (FL) has been widely studied recently due to its property to collaboratively train data from different devices without sharing the raw data. Nevertheless, recent studies show that an adversary can still be possible to infer private information about devices' data, e.g., sensitive attributes such as income, race, and sexual orientation. To mitigate the attribute inference attacks, various existing privacy-preserving FL methods can be adopted/adapted. However, all these existing methods have key limitations: they need to know the FL task in advance, or have intolerable computational overheads or utility losses, or do not have provable privacy guarantees. We address these issues and design a task-agnostic privacy-preserving presentation learning method for FL ({\bf TAPPFL}) against attribute inference attacks. TAPPFL is formulated via information theory. Specifically, TAPPFL has two mutual information goals, where one goal learns task-agnostic data representations that contain the least information about the private attribute in each device's data, and the other goal ensures the learnt data representations include as much information as possible about the device data to maintain FL utility. We also derive privacy guarantees of TAPPFL against worst-case attribute inference attacks, as well as the inherent tradeoff between utility preservation and privacy protection. Extensive results on multiple datasets and applications validate the effectiveness of TAPPFL to protect data privacy, maintain the FL utility, and be efficient as well. Experimental results also show that TAPPFL outperforms the existing defenses\footnote{Source code and full version: \url{https://github.com/TAPPFL}}.