FedRE: Robust and Effective Federated Learning with Privacy Preference

TL;DR

FedRE introduces a privacy-aware federated learning framework that allocates privacy budgets based on client-specific privacy preferences, enhancing robustness and effectiveness while maintaining privacy guarantees.

Contribution

The paper proposes a novel method to incorporate privacy preferences into federated learning, optimizing local differential privacy to reduce unnecessary noise and improve model performance.

Findings

FedRE achieves competitive results on text tamper detection datasets.

Privacy budget allocation based on PSI improves model robustness.

Method effectively balances privacy protection and model accuracy.

Abstract

Despite Federated Learning (FL) employing gradient aggregation at the server for distributed training to prevent the privacy leakage of raw data, private information can still be divulged through the analysis of uploaded gradients from clients. Substantial efforts have been made to integrate local differential privacy (LDP) into the system to achieve a strict privacy guarantee. However, existing methods fail to take practical issues into account by merely perturbing each sample with the same mechanism while each client may have their own privacy preferences on privacy-sensitive information (PSI), which is not uniformly distributed across the raw data. In such a case, excessive privacy protection from private-insensitive information can additionally introduce unnecessary noise, which may degrade the model performance. In this work, we study the PSI within data and develop FedRE, that can simultaneously achieve robustness and effectiveness benefits with LDP protection. More specifically, we first define PSI with regard to the privacy preferences of each client. Then, we optimize the LDP by allocating less privacy budget to gradients with higher PSI in a layer-wise manner, thus providing a stricter privacy guarantee for PSI. Furthermore, to mitigate the performance degradation caused by LDP, we design a parameter aggregation mechanism based on the distribution of the perturbed information. We conducted experiments with text tamper detection on T-SROIE and DocTamper datasets, and FedRE achieves competitive performance compared to state-of-the-art methods.