A Unified Convergence Analysis for Semi-Decentralized Learning: Sampled-to-Sampled vs. Sampled-to-All Communication

TL;DR

This paper provides a unified convergence analysis comparing sampled-to-sampled and sampled-to-all communication strategies in semi-decentralized federated learning, offering theoretical insights and practical guidelines based on system parameters and data heterogeneity.

Contribution

It introduces a unified framework for analyzing semi-decentralized federated learning, comparing two communication strategies and deriving conditions for their effectiveness.

Findings

Sampled-to-all outperforms in low heterogeneity scenarios.

Sampled-to-sampled is better when data heterogeneity is high.

Guidelines for choosing communication strategies based on system parameters.

Abstract

In semi-decentralized federated learning, devices primarily rely on device-to-device communication but occasionally interact with a central server. Periodically, a sampled subset of devices uploads their local models to the server, which computes an aggregate model. The server can then either (i) share this aggregate model only with the sampled clients (sampled-to-sampled, S2S) or (ii) broadcast it to all clients (sampled-to-all, S2A). Despite their practical significance, a rigorous theoretical and empirical comparison of these two strategies remains absent. We address this gap by analyzing S2S and S2A within a unified convergence framework that accounts for key system parameters: sampling rate, server aggregation frequency, and network connectivity. Our results, both analytical and experimental, reveal distinct regimes where one strategy outperforms the other, depending primarily on the degree of data heterogeneity across devices. These insights lead to concrete design guidelines for practical semi-decentralized FL deployments.