An Experimental Study of Class Imbalance in Federated Learning

TL;DR

This paper investigates how class imbalance affects federated learning, introducing new metrics to measure imbalance and analyzing their impact on global model performance through extensive experiments.

Contribution

It proposes two novel metrics for class imbalance in federated learning and provides an empirical analysis of their effects on model performance and convergence.

Findings

Higher MID and WCS worsen global model accuracy.

WCS slows down convergence by misdirecting optimization.

Class imbalance metrics significantly influence federated learning outcomes.

Abstract

Federated learning is a distributed machine learning paradigm that trains a global model for prediction based on a number of local models at clients while local data privacy is preserved. Class imbalance is believed to be one of the factors that degrades the global model performance. However, there has been very little research on if and how class imbalance can affect the global performance. class imbalance in federated learning is much more complex than that in traditional non-distributed machine learning, due to different class imbalance situations at local clients. Class imbalance needs to be re-defined in distributed learning environments. In this paper, first, we propose two new metrics to define class imbalance -- the global class imbalance degree (MID) and the local difference of class imbalance among clients (WCS). Then, we conduct extensive experiments to analyze the impact of class imbalance on the global performance in various scenarios based on our definition. Our results show that a higher MID and a larger WCS degrade more the performance of the global model. Besides, WCS is shown to slow down the convergence of the global model by misdirecting the optimization.