FedSat: A Statistical Aggregation Approach for Class Imbalanced Clients in Federated Learning

TL;DR

FedSat is a federated learning method that effectively handles data heterogeneity across clients by using a prediction-sensitive loss and a weighted aggregation scheme, leading to improved accuracy and faster convergence.

Contribution

FedSat introduces a novel approach combining a prediction-sensitive loss and prioritized-class weighted aggregation to address multiple data heterogeneity issues in federated learning.

Findings

Outperforms state-of-the-art baselines by 1.8% on average

Achieves 19.87% improvement over weakest baselines

Demonstrates faster convergence than existing methods

Abstract

Federated learning (FL) has emerged as a promising paradigm for privacy-preserving distributed machine learning, but faces challenges with heterogeneous data distributions across clients. This paper presents FedSat, a novel FL approach specifically designed to simultaneously handle three forms of data heterogeneity, namely label skewness, missing classes, and quantity skewness, by proposing a prediction-sensitive loss function and a prioritized-class based weighted aggregation scheme. While the prediction-sensitive loss function enhances model performance on minority classes, the prioritized-class based weighted aggregation scheme ensures client contributions are weighted based on both statistical significance and performance on critical classes. Extensive experiments across diverse data-heterogeneity settings demonstrate that FedSat significantly outperforms state-of-the-art baselines, with an average improvement of 1.8% over the second-best method and 19.87% over the weakest-performing baseline. The approach also demonstrates faster convergence compared to existing methods. These results highlight FedSat's effectiveness in addressing the challenges of heterogeneous federated learning and its potential for real-world applications.