Local Gradient Regulation Stabilizes Federated Learning under Client Heterogeneity

TL;DR

This paper identifies that client heterogeneity destabilizes federated learning by distorting local gradients and proposes a gradient regulation method, ECGR, to stabilize training without extra communication, validated through theoretical and experimental results.

Contribution

It introduces a novel client-side gradient regulation perspective and the ECGR method to enhance FL stability under data heterogeneity.

Findings

ECGR stabilizes federated learning across various methods.

Regulating local gradients improves convergence in heterogeneous settings.

Experimental validation on medical imaging dataset supports effectiveness.

Abstract

Federated learning (FL) enables collaborative model training across distributed clients without sharing raw data, yet its stability is fundamentally challenged by statistical heterogeneity in realistic deployments. Here, we show that client heterogeneity destabilizes FL primarily by distorting local gradient dynamics during client-side optimization, causing systematic drift that accumulates across communication rounds and impedes global convergence. This observation highlights local gradients as a key regulatory lever for stabilizing heterogeneous FL systems. Building on this insight, we develop a general client-side perspective that regulates local gradient contributions without incurring additional communication overhead. Inspired by swarm intelligence, we instantiate this perspective through Exploratory--Convergent Gradient Re-aggregation (ECGR), which balances well-aligned and misaligned gradient components to preserve informative updates while suppressing destabilizing effects. Theoretical analysis and extensive experiments, including evaluations on the LC25000 medical imaging dataset, demonstrate that regulating local gradient dynamics consistently stabilizes federated learning across state-of-the-art methods under heterogeneous data distributions.