SPEAR++: Scaling Gradient Inversion via Sparsely-Used Dictionary Learning

TL;DR

SPEAR++ advances gradient inversion attacks in federated learning by applying sparsely-used dictionary learning, enabling practical attacks on larger batch sizes while maintaining robustness against defenses.

Contribution

This work extends SPEAR by integrating dictionary learning techniques, making gradient inversion attacks scalable and practical for larger batch sizes in federated learning.

Findings

Retains robustness to differential privacy noise

Applicable to 10x larger batch sizes

Demonstrates improved attack practicality

Abstract

Federated Learning has seen an increased deployment in real-world scenarios recently, as it enables the distributed training of machine learning models without explicit data sharing between individual clients. Yet, the introduction of the so-called gradient inversion attacks has fundamentally challenged its privacy-preserving properties. Unfortunately, as these attacks mostly rely on direct data optimization without any formal guarantees, the vulnerability of real-world systems remains in dispute and requires tedious testing for each new federated deployment. To overcome these issues, recently the SPEAR attack was introduced, which is based on a theoretical analysis of the gradients of linear layers with ReLU activations. While SPEAR is an important theoretical breakthrough, the attack's practicality was severely limited by its exponential runtime in the batch size b. In this work, we fill this gap by applying State-of-the-Art techniques from Sparsely-Used Dictionary Learning to make the problem of gradient inversion on linear layers with ReLU activations tractable. Our experiments demonstrate that our new attack, SPEAR++, retains all desirable properties of SPEAR, such as robustness to DP noise and FedAvg aggregation, while being applicable to 10x bigger batch sizes.