A Unified Learn-to-Distort-Data Framework for Privacy-Utility Trade-off in Trustworthy Federated Learning

TL;DR

This paper introduces a unified framework for balancing privacy and utility in federated learning by learning optimal data distortions, supported by theoretical foundations and connections to related areas.

Contribution

The paper proposes the Learn-to-Distort-Data framework, a novel approach to explicitly model and optimize data distortion for privacy-utility trade-offs in federated learning.

Findings

Framework effectively balances privacy and utility.

Connections to adversarial training and robustness enhance algorithm design.

Applicable to various privacy-preserving mechanisms.

Abstract

In this paper, we first give an introduction to the theoretical basis of the privacy-utility equilibrium in federated learning based on Bayesian privacy definitions and total variation distance privacy definitions. We then present the \textit{Learn-to-Distort-Data} framework, which provides a principled approach to navigate the privacy-utility equilibrium by explicitly modeling the distortion introduced by the privacy-preserving mechanism as a learnable variable and optimizing it jointly with the model parameters. We demonstrate the applicability of our framework to a variety of privacy-preserving mechanisms on the basis of data distortion and highlight its connections to related areas such as adversarial training, input robustness, and unlearnable examples. These connections enable leveraging techniques from these areas to design effective algorithms for privacy-utility equilibrium in federated learning under the \textit{Learn-to-Distort-Data} framework.