A Communication Efficient Collaborative Learning Framework for Distributed Features

TL;DR

This paper presents a communication-efficient federated learning framework for distributed features, introducing FedBCD with local updates to reduce communication rounds while maintaining convergence guarantees.

Contribution

The paper proposes FedBCD, a novel federated learning algorithm with multiple local updates, reducing communication rounds and providing theoretical convergence analysis.

Findings

FedBCD reduces communication rounds compared to traditional methods.

Theoretical analysis shows convergence within O(√T) communication rounds.

Empirical results demonstrate advantages over standard SGD approaches.

Abstract

We introduce a collaborative learning framework allowing multiple parties having different sets of attributes about the same user to jointly build models without exposing their raw data or model parameters. In particular, we propose a Federated Stochastic Block Coordinate Descent (FedBCD) algorithm, in which each party conducts multiple local updates before each communication to effectively reduce the number of communication rounds among parties, a principal bottleneck for collaborative learning problems. We analyze theoretically the impact of the number of local updates and show that when the batch size, sample size, and the local iterations are selected appropriately, within $T$ iterations, the algorithm performs $\mathcal{O}(\sqrt{T})$ communication rounds and achieves some $\mathcal{O}(1/\sqrt{T})$ accuracy (measured by the average of the gradient norm squared). The approach is supported by our empirical evaluations on a variety of tasks and datasets, demonstrating advantages over stochastic gradient descent (SGD) approaches.