Differentially Private Federated Learning: Servers Trustworthiness, Estimation, and Statistical Inference

TL;DR

This paper explores the challenges and solutions for privacy-preserving federated learning in high-dimensional settings, focusing on estimation accuracy, inference methods, and the impact of server trustworthiness.

Contribution

It introduces new estimation and inference algorithms for differentially private federated learning, addressing both trusted and untrusted server scenarios in high-dimensional linear models.

Findings

Estimation rates depend on data dimensionality even with sparsity.

Proposed algorithms enable effective statistical inference under privacy constraints.

Simulation results validate the theoretical guarantees and practical effectiveness.

Abstract

Differentially private federated learning is crucial for maintaining privacy in distributed environments. This paper investigates the challenges of high-dimensional estimation and inference under the constraints of differential privacy. First, we study scenarios involving an untrusted central server, demonstrating the inherent difficulties of accurate estimation in high-dimensional problems. Our findings indicate that the tight minimax rates depends on the high-dimensionality of the data even with sparsity assumptions. Second, we consider a scenario with a trusted central server and introduce a novel federated estimation algorithm tailored for linear regression models. This algorithm effectively handles the slight variations among models distributed across different machines. We also propose methods for statistical inference, including coordinate-wise confidence intervals for individual parameters and strategies for simultaneous inference. Extensive simulation experiments support our theoretical advances, underscoring the efficacy and reliability of our approaches.