Sharp Gaussian approximations for Decentralized Federated Learning

TL;DR

This paper develops Gaussian approximation results for local SGD in decentralized federated learning, enabling statistical inference and robustness testing.

Contribution

It introduces Berry-Esseen and time-uniform Gaussian approximations for local SGD, extending asymptotic guarantees beyond convergence.

Findings

Berry-Esseen theorem for local SGD iterates

Time-uniform Gaussian approximations for entire trajectories

Simulations validating theoretical results

Abstract

Federated Learning has gained traction in privacy-sensitive collaborative environments, with local SGD emerging as a key optimization method in decentralized settings. While its convergence properties are well-studied, asymptotic statistical guarantees beyond convergence remain limited. In this paper, we present two generalized Gaussian approximation results for local SGD and explore their implications. First, we prove a Berry-Esseen theorem for the final local SGD iterates, enabling valid multiplier bootstrap procedures. Second, motivated by robustness considerations, we introduce two distinct time-uniform Gaussian approximations for the entire trajectory of local SGD. The time-uniform approximations support Gaussian bootstrap-based tests for detecting adversarial attacks. Extensive simulations are provided to support our theoretical results.