QBI: Quantile-Based Bias Initialization for Efficient Private Data Reconstruction in Federated Learning

TL;DR

This paper introduces QBI, a bias initialization method that enhances data reconstruction in federated learning, along with PAIRS and AGGP, to improve attack success and defend against gradient sparsity attacks, respectively.

Contribution

The paper presents QBI and PAIRS for improved data reconstruction in federated learning, and proposes AGGP as a defense against gradient sparsity attacks, advancing both attack and defense strategies.

Findings

QBI improves reconstruction accuracy by up to 50% on ImageNet.

PAIRS increases the percentage of fully reconstructed data.

AGGP effectively prevents gradient sparsity attacks.

Abstract

Federated learning enables the training of machine learning models on distributed data without compromising user privacy, as data remains on personal devices and only model updates, such as gradients, are shared with a central coordinator. However, recent research has shown that the central entity can perfectly reconstruct private data from shared model updates by maliciously initializing the model's parameters. In this paper, we propose QBI, a novel bias initialization method that significantly enhances reconstruction capabilities. This is accomplished by directly solving for bias values yielding sparse activation patterns. Further, we propose PAIRS, an algorithm that builds on QBI. PAIRS can be deployed when a separate dataset from the target domain is available to further increase the percentage of data that can be fully recovered. Measured by the percentage of samples that can be perfectly reconstructed from batches of various sizes, our approach achieves significant improvements over previous methods with gains of up to 50% on ImageNet and up to 60% on the IMDB sentiment analysis text dataset. Furthermore, we establish theoretical limits for attacks leveraging stochastic gradient sparsity, providing a foundation for understanding the fundamental constraints of these attacks. We empirically assess these limits using synthetic datasets. Finally, we propose and evaluate AGGP, a defensive framework designed to prevent gradient sparsity attacks, contributing to the development of more secure and private federated learning systems.