Federated Domain Generalization with Label Smoothing and Balanced Decentralized Training

TL;DR

This paper introduces FedSB, a federated learning method that uses label smoothing and decentralized training to improve domain generalization across diverse datasets, outperforming existing methods.

Contribution

FedSB is a novel federated learning approach combining label smoothing and decentralized training to enhance domain generalization and performance across heterogeneous data sources.

Findings

FedSB outperforms competing methods on three of four datasets.

It effectively prevents overfitting to domain-specific features.

The decentralized budgeting mechanism improves global model performance.

Abstract

In this paper, we propose a novel approach, Federated Domain Generalization with Label Smoothing and Balanced Decentralized Training (FedSB), to address the challenges of data heterogeneity within a federated learning framework. FedSB utilizes label smoothing at the client level to prevent overfitting to domain-specific features, thereby enhancing generalization capabilities across diverse domains when aggregating local models into a global model. Additionally, FedSB incorporates a decentralized budgeting mechanism which balances training among clients, which is shown to improve the performance of the aggregated global model. Extensive experiments on four commonly used multi-domain datasets, PACS, VLCS, OfficeHome, and TerraInc, demonstrate that FedSB outperforms competing methods, achieving state-of-the-art results on three out of four datasets, indicating the effectiveness of FedSB in addressing data heterogeneity.