General Lower Bounds for Differentially Private Federated Learning with Arbitrary Public-Transcript Interactions

TL;DR

This paper establishes a fundamental lower bound for differentially private federated learning protocols, accommodating arbitrary public-transcript interactions and adaptive rounds, with applications to various estimation tasks.

Contribution

It introduces a general lower bound for private federated learning that accounts for complex interactions and reusing local data across rounds, advancing theoretical understanding.

Findings

Derived a federated van Trees lower bound for parameter estimation.

Applied the bound to mean estimation, linear regression, and nonparametric regression.

Developed a privacy-information contraction inequality for public transcripts.

Abstract

We prove a general lower bound for differentially private federated learning protocols with arbitrary public-transcript interactions. The protocol may use any number of adaptive rounds, and each client's local samples may be reused across rounds. For parameter estimation under squared \(\ell_2\) loss, we establish a federated van Trees lower bound for every estimator satisfying a total clientwise sample-level zero-concentrated differential privacy (zCDP) constraint. The main technical ingredient is a privacy-information contraction inequality for complete public transcripts. We illustrate the bound through applications to mean estimation, linear regression, and nonparametric regression.