Optimizing the Collaboration Structure in Cross-Silo Federated Learning

TL;DR

This paper introduces FedCollab, a federated learning framework that clusters clients based on data similarity to reduce negative transfer and improve model performance in non-IID settings.

Contribution

FedCollab is a novel clustering-based FL framework that dynamically groups clients to mitigate negative transfer caused by data heterogeneity.

Findings

FedCollab effectively reduces negative transfer across various datasets and models.

It outperforms existing clustered FL algorithms in diverse non-IID scenarios.

The framework improves global model accuracy in federated learning.

Abstract

In federated learning (FL), multiple clients collaborate to train machine learning models together while keeping their data decentralized. Through utilizing more training data, FL suffers from the potential negative transfer problem: the global FL model may even perform worse than the models trained with local data only. In this paper, we propose FedCollab, a novel FL framework that alleviates negative transfer by clustering clients into non-overlapping coalitions based on their distribution distances and data quantities. As a result, each client only collaborates with the clients having similar data distributions, and tends to collaborate with more clients when it has less data. We evaluate our framework with a variety of datasets, models, and types of non-IIDness. Our results demonstrate that FedCollab effectively mitigates negative transfer across a wide range of FL algorithms and consistently outperforms other clustered FL algorithms.