LDP-Fed: Federated Learning with Local Differential Privacy

TL;DR

LDP-Fed introduces a federated learning system that guarantees local differential privacy for high-dimensional model updates, enabling privacy-preserving training of neural networks across multiple participants.

Contribution

It develops two novel approaches for applying local differential privacy to iterative, high-dimensional model updates in federated learning, addressing limitations of existing LDP protocols.

Findings

LDP-Fed achieves formal privacy guarantees for federated neural network training.

CLDP-Fed outperforms existing methods in model accuracy and privacy preservation.

The system effectively filters and perturbs model updates to maintain privacy without sacrificing performance.

Abstract

This paper presents LDP-Fed, a novel federated learning system with a formal privacy guarantee using local differential privacy (LDP). Existing LDP protocols are developed primarily to ensure data privacy in the collection of single numerical or categorical values, such as click count in Web access logs. However, in federated learning model parameter updates are collected iteratively from each participant and consist of high dimensional, continuous values with high precision (10s of digits after the decimal point), making existing LDP protocols inapplicable. To address this challenge in LDP-Fed, we design and develop two novel approaches. First, LDP-Fed's LDP Module provides a formal differential privacy guarantee for the repeated collection of model training parameters in the federated training of large-scale neural networks over multiple individual participants' private datasets. Second, LDP-Fed implements a suite of selection and filtering techniques for perturbing and sharing select parameter updates with the parameter server. We validate our system deployed with a condensed LDP protocol in training deep neural networks on public data. We compare this version of LDP-Fed, coined CLDP-Fed, with other state-of-the-art approaches with respect to model accuracy, privacy preservation, and system capabilities.