Composite federated learning with heterogeneous data

TL;DR

This paper introduces a new federated learning algorithm that efficiently handles heterogeneous data and non-smooth regularization, reducing communication costs and ensuring convergence without data similarity assumptions.

Contribution

The proposed algorithm decouples the proximal operator from communication, manages non-smooth regularization, and reduces communication by local updates, with proven linear convergence.

Findings

Outperforms state-of-the-art methods in numerical experiments

Converges linearly to a neighborhood of the optimal solution

Effectively handles heterogeneous data without similarity assumptions

Abstract

We propose a novel algorithm for solving the composite Federated Learning (FL) problem. This algorithm manages non-smooth regularization by strategically decoupling the proximal operator and communication, and addresses client drift without any assumptions about data similarity. Moreover, each worker uses local updates to reduce the communication frequency with the server and transmits only a $d$-dimensional vector per communication round. We prove that our algorithm converges linearly to a neighborhood of the optimal solution and demonstrate the superiority of our algorithm over state-of-the-art methods in numerical experiments.