iDLG: Improved Deep Leakage from Gradients

TL;DR

This paper introduces iDLG, an improved method for extracting private training data and ground-truth labels from shared gradients in distributed learning, demonstrating higher accuracy and reliability than previous approaches.

Contribution

The paper presents a new approach, iDLG, that reliably extracts ground-truth labels from gradients, overcoming convergence issues of previous methods like DLG.

Findings

iDLG accurately extracts ground-truth labels from gradients.

iDLG outperforms DLG in convergence and label recovery.

The method is applicable to models trained with cross-entropy loss over one-hot labels.

Abstract

It is widely believed that sharing gradients will not leak private training data in distributed learning systems such as Collaborative Learning and Federated Learning, etc. Recently, Zhu et al. presented an approach which shows the possibility to obtain private training data from the publicly shared gradients. In their Deep Leakage from Gradient (DLG) method, they synthesize the dummy data and corresponding labels with the supervision of shared gradients. However, DLG has difficulty in convergence and discovering the ground-truth labels consistently. In this paper, we find that sharing gradients definitely leaks the ground-truth labels. We propose a simple but reliable approach to extract accurate data from the gradients. Particularly, our approach can certainly extract the ground-truth labels as opposed to DLG, hence we name it Improved DLG (iDLG). Our approach is valid for any differentiable model trained with cross-entropy loss over one-hot labels. We mathematically illustrate how our method can extract ground-truth labels from the gradients and empirically demonstrate the advantages over DLG.