MultiConfederated Learning: Inclusive Non-IID Data handling with Decentralized Federated Learning

TL;DR

This paper introduces MultiConfederated Learning, a decentralized federated learning framework that maintains multiple models in parallel to effectively handle non-IID data and improve convergence without relying on a central server.

Contribution

The paper proposes a novel decentralized FL approach that maintains multiple models and uses transfer learning to address non-IID data challenges, enhancing robustness and adaptability.

Findings

Improved model convergence with non-IID data

Decentralized framework reduces single point of failure

Learners selectively aggregate peer updates

Abstract

Federated Learning (FL) has emerged as a prominent privacy-preserving technique for enabling use cases like confidential clinical machine learning. FL operates by aggregating models trained by remote devices which owns the data. Thus, FL enables the training of powerful global models using crowd-sourced data from a large number of learners, without compromising their privacy. However, the aggregating server is a single point of failure when generating the global model. Moreover, the performance of the model suffers when the data is not independent and identically distributed (non-IID data) on all remote devices. This leads to vastly different models being aggregated, which can reduce the performance by as much as 50% in certain scenarios. In this paper, we seek to address the aforementioned issues while retaining the benefits of FL. We propose MultiConfederated Learning: a decentralized FL framework which is designed to handle non-IID data. Unlike traditional FL, MultiConfederated Learning will maintain multiple models in parallel (instead of a single global model) to help with convergence when the data is non-IID. With the help of transfer learning, learners can converge to fewer models. In order to increase adaptability, learners are allowed to choose which updates to aggregate from their peers.