FedRD: Reducing Divergences for Generalized Federated Learning via Heterogeneity-aware Parameter Guidance

TL;DR

FedRD is a novel federated learning algorithm designed to improve model generalization across heterogeneous clients by reducing divergences through parameter guidance and local debiasing, showing superior performance on multi-domain datasets.

Contribution

The paper introduces FedRD, a heterogeneity-aware federated learning method that addresses optimization and performance divergences for better generalization to unseen clients.

Findings

FedRD outperforms baseline methods on multi-domain datasets.

It effectively reduces optimization and performance divergences.

The approach enhances model generalization in heterogeneous federated settings.

Abstract

Heterogeneous federated learning (HFL) aims to ensure effective and privacy-preserving collaboration among different entities. As newly joined clients require significant adjustments and additional training to align with the existing system, the problem of generalizing federated learning models to unseen clients under heterogeneous data has become progressively crucial. Consequently, we highlight two unsolved challenging issues in federated domain generalization: Optimization Divergence and Performance Divergence. To tackle the above challenges, we propose FedRD, a novel heterogeneity-aware federated learning algorithm that collaboratively utilizes parameter-guided global generalization aggregation and local debiased classification to reduce divergences, aiming to obtain an optimal global model for participating and unseen clients. Extensive experiments on public multi-domain datasets demonstrate that our approach exhibits a substantial performance advantage over competing baselines in addressing this specific problem.