Semi-Decentralized Federated Learning with Collaborative Relaying

TL;DR

This paper introduces a semi-decentralized federated learning algorithm where clients collaborate through relaying neighbor updates to a central server, optimizing weights to improve convergence and accuracy, especially under intermittent connectivity.

Contribution

The paper proposes a novel semi-decentralized federated learning method with optimized neighbor-based averaging, enhancing convergence and robustness over traditional federated averaging.

Findings

Improved convergence rate compared to federated averaging.

Enhanced accuracy in scenarios with intermittent connectivity.

Theoretical analysis supporting the unbiasedness and variance reduction.

Abstract

We present a semi-decentralized federated learning algorithm wherein clients collaborate by relaying their neighbors' local updates to a central parameter server (PS). At every communication round to the PS, each client computes a local consensus of the updates from its neighboring clients and eventually transmits a weighted average of its own update and those of its neighbors to the PS. We appropriately optimize these averaging weights to ensure that the global update at the PS is unbiased and to reduce the variance of the global update at the PS, consequently improving the rate of convergence. Numerical simulations substantiate our theoretical claims and demonstrate settings with intermittent connectivity between the clients and the PS, where our proposed algorithm shows an improved convergence rate and accuracy in comparison with the federated averaging algorithm.