Measuring Lower Bounds of Local Differential Privacy via Adversary Instantiations in Federated Learning

TL;DR

This paper introduces an empirical privacy test to measure the lower bounds of local differential privacy in federated learning, using adversary models to interpret privacy levels and evaluate attack surfaces.

Contribution

It presents a novel empirical approach with adversary instantiations to interpret and measure LDP in federated learning, including worst-case attack analysis.

Findings

Empirical privacy bounds can be effectively measured using adversary models.

Worst-case attack bounds are not realistic in practical federated learning scenarios.

Relaxation of privacy parameters is possible without compromising security in certain attack models.

Abstract

Local differential privacy (LDP) gives a strong privacy guarantee to be used in a distributed setting like federated learning (FL). LDP mechanisms in FL protect a client's gradient by randomizing it on the client; however, how can we interpret the privacy level given by the randomization? Moreover, what types of attacks can we mitigate in practice? To answer these questions, we introduce an empirical privacy test by measuring the lower bounds of LDP. The privacy test estimates how an adversary predicts if a reported randomized gradient was crafted from a raw gradient $g_1$ or $g_2$. We then instantiate six adversaries in FL under LDP to measure empirical LDP at various attack surfaces, including a worst-case attack that reaches the theoretical upper bound of LDP. The empirical privacy test with the adversary instantiations enables us to interpret LDP more intuitively and discuss relaxation of the privacy parameter until a particular instantiated attack surfaces. We also demonstrate numerical observations of the measured privacy in these adversarial settings, and the worst-case attack is not realistic in FL. In the end, we also discuss the possible relaxation of privacy levels in FL under LDP.