FedDPC : Handling Data Heterogeneity and Partial Client Participation in Federated Learning

TL;DR

FedDPC is a federated learning method that effectively reduces the impact of data heterogeneity and partial client participation, leading to faster training and better model accuracy.

Contribution

The paper introduces FedDPC, a novel approach that projects local updates onto previous global updates and employs adaptive scaling to handle data heterogeneity and partial participation.

Findings

FedDPC achieves faster training loss reduction.

FedDPC improves test accuracy over state-of-the-art methods.

FedDPC demonstrates robustness across multiple datasets.

Abstract

Data heterogeneity is a significant challenge in modern federated learning (FL) as it creates variance in local model updates, causing the aggregated global model to shift away from the true global optimum. Partial client participation in FL further exacerbates this issue by skewing the aggregation of local models towards the data distribution of participating clients. This creates additional variance in the global model updates, causing the global model to converge away from the optima of the global objective. These variances lead to instability in FL training, which degrades global model performance and slows down FL training. While existing literature primarily focuses on addressing data heterogeneity, the impact of partial client participation has received less attention. In this paper, we propose FedDPC, a novel FL method, designed to improve FL training and global model performance by mitigating both data heterogeneity and partial client participation. FedDPC addresses these issues by projecting each local update onto the previous global update, thereby controlling variance in both local and global updates. To further accelerate FL training, FedDPC employs adaptive scaling for each local update before aggregation. Extensive experiments on image classification tasks with multiple heterogeneously partitioned datasets validate the effectiveness of FedDPC. The results demonstrate that FedDPC outperforms state-of-the-art FL algorithms by achieving faster reduction in training loss and improved test accuracy across communication rounds.