Kuramoto-FedAvg: Using Synchronization Dynamics to Improve Federated Learning Optimization under Statistical Heterogeneity

TL;DR

Kuramoto-FedAvg introduces a synchronization-based approach inspired by coupled oscillators to enhance federated learning convergence under non-IID data, effectively reducing client drift and accelerating training.

Contribution

This work proposes a novel federated optimization algorithm that uses synchronization dynamics to improve convergence in heterogeneous data settings.

Findings

Accelerates convergence compared to FedAvg.

Reduces client drift through phase-based weighting.

Improves accuracy on benchmark datasets.

Abstract

Federated learning on heterogeneous (non-IID) client data experiences slow convergence due to client drift. To address this challenge, we propose Kuramoto-FedAvg, a federated optimization algorithm that reframes the weight aggregation step as a synchronization problem inspired by the Kuramoto model of coupled oscillators. The server dynamically weighs each client's update based on its phase alignment with the global update, amplifying contributions that align with the global gradient direction while minimizing the impact of updates that are out of phase. We theoretically prove that this synchronization mechanism reduces client drift, providing a tighter convergence bound compared to the standard FedAvg under heterogeneous data distributions. Empirical validation supports our theoretical findings, showing that Kuramoto-FedAvg significantly accelerates convergence and improves accuracy across multiple benchmark datasets. Our work highlights the potential of coordination and synchronization-based strategies for managing gradient diversity and accelerating federated optimization in realistic non-IID settings.