Federated Learning Beyond the Star: Local D2D Model Consensus with Global Cluster Sampling

TL;DR

This paper introduces TT-HF, a hybrid federated learning approach that incorporates device-to-device communications and cluster sampling to enhance convergence and efficiency beyond traditional star-topology federated learning.

Contribution

The paper proposes a novel two-timescale hybrid federated learning framework that integrates D2D communications and cluster sampling, with theoretical convergence guarantees and improved performance.

Findings

TT-HF achieves faster convergence than baseline methods.

Theoretical analysis provides conditions for O(1/t) convergence rate.

Experimental results show improved model utilization and convergence speed.

Abstract

Federated learning has emerged as a popular technique for distributing model training across the network edge. Its learning architecture is conventionally a star topology between the devices and a central server. In this paper, we propose two timescale hybrid federated learning (TT-HF), which migrates to a more distributed topology via device-to-device (D2D) communications. In TT-HF, local model training occurs at devices via successive gradient iterations, and the synchronization process occurs at two timescales: (i) macro-scale, where global aggregations are carried out via device-server interactions, and (ii) micro-scale, where local aggregations are carried out via D2D cooperative consensus formation in different device clusters. Our theoretical analysis reveals how device, cluster, and network-level parameters affect the convergence of TT-HF, and leads to a set of conditions under which a convergence rate of O(1/t) is guaranteed. Experimental results demonstrate the improvements in convergence and utilization that can be obtained by TT-HF over state-of-the-art federated learning baselines.