On the Convergence of FedProx with Extrapolation and Inexact Prox

TL;DR

This paper analyzes the convergence of the FedExProx federated learning algorithm with server-side extrapolation when clients compute proximal operators approximately, showing it converges to a neighborhood of the solution and can be made robust to inexactness.

Contribution

It extends the theoretical analysis of FedExProx by removing the exactness assumption, demonstrating convergence behavior with inexact proximal computations in convex settings.

Findings

Inexactness causes convergence to a neighborhood of the solution.

Careful control of inexactness mitigates adverse effects.

Robustness of extrapolation to inexact proximal updates is validated.

Abstract

Enhancing the FedProx federated learning algorithm (Li et al., 2020) with server-side extrapolation, Li et al. (2024a) recently introduced the FedExProx method. Their theoretical analysis, however, relies on the assumption that each client computes a certain proximal operator exactly, which is impractical since this is virtually never possible to do in real settings. In this paper, we investigate the behavior of FedExProx without this exactness assumption in the smooth and globally strongly convex setting. We establish a general convergence result, showing that inexactness leads to convergence to a neighborhood of the solution. Additionally, we demonstrate that, with careful control, the adverse effects of this inexactness can be mitigated. By linking inexactness to biased compression (Beznosikov et al., 2023), we refine our analysis, highlighting robustness of extrapolation to inexact proximal updates. We also examine the local iteration complexity required by each client to achieved the required level of inexactness using various local optimizers. Our theoretical insights are validated through comprehensive numerical experiments.