SGFusion: Stochastic Geographic Gradient Fusion in Federated Learning

TL;DR

SGFusion introduces a probabilistic, geographically-aware federated learning algorithm that enhances model accuracy by fusing gradients across similar zones, leveraging hierarchical random graphs and stochastic attention mechanisms.

Contribution

The paper presents SGFusion, a novel FL training method that incorporates geographic information and probabilistic gradient fusion using hierarchical random graphs and self-attention.

Findings

Improves model utility across diverse geographic zones.

Converges with bounded expected errors in empirical tests.

Achieves significant accuracy gains without extra computational cost.

Abstract

This paper proposes Stochastic Geographic Gradient Fusion (SGFusion), a novel training algorithm to leverage the geographic information of mobile users in Federated Learning (FL). SGFusion maps the data collected by mobile devices onto geographical zones and trains one FL model per zone, which adapts well to the data and behaviors of users in that zone. SGFusion models the local data-based correlation among geographical zones as a hierarchical random graph (HRG) optimized by Markov Chain Monte Carlo sampling. At each training step, every zone fuses its local gradient with gradients derived from a small set of other zones sampled from the HRG. This approach enables knowledge fusion and sharing among geographical zones in a probabilistic and stochastic gradient fusion process with self-attention weights, such that "more similar" zones have "higher probabilities" of sharing gradients with "larger attention weights." SGFusion remarkably improves model utility without introducing undue computational cost. Extensive theoretical and empirical results using a heart-rate prediction dataset collected across 6 countries show that models trained with SGFusion converge with upper-bounded expected errors and significantly improve utility in all countries compared to existing approaches without notable cost in system scalability.