Uncovering Gradient Inversion Risks in Practical Language Model Training

TL;DR

This paper introduces Grab, a novel gradient inversion attack tailored for language models in federated learning, demonstrating high data recovery rates and revealing significant privacy risks.

Contribution

It presents a domain-specific attack method that overcomes practical training challenges, significantly improving data recovery in language model privacy attacks.

Findings

Recover up to 92.9% of private data

Outperform existing methods by up to 28.9% in benchmark settings

Achieve 48.5% higher recovery in practical scenarios

Abstract

The gradient inversion attack has been demonstrated as a significant privacy threat to federated learning (FL), particularly in continuous domains such as vision models. In contrast, it is often considered less effective or highly dependent on impractical training settings when applied to language models, due to the challenges posed by the discrete nature of tokens in text data. As a result, its potential privacy threats remain largely underestimated, despite FL being an emerging training method for language models. In this work, we propose a domain-specific gradient inversion attack named Grab (gradient inversion with hybrid optimization). Grab features two alternating optimization processes to address the challenges caused by practical training settings, including a simultaneous optimization on dropout masks between layers for improved token recovery and a discrete optimization for effective token sequencing. Grab can recover a significant portion (up to 92.9% recovery rate) of the private training data, outperforming the attack strategy of utilizing discrete optimization with an auxiliary model by notable improvements of up to 28.9% recovery rate in benchmark settings and 48.5% recovery rate in practical settings. Grab provides a valuable step forward in understanding this privacy threat in the emerging FL training mode of language models.