Eliminating Domain Bias for Federated Learning in Representation Space

TL;DR

This paper introduces a framework called Domain Bias Eliminator (DBE) that reduces domain bias in federated learning, improving model generalization and personalization across heterogeneous client data.

Contribution

The paper presents a novel framework that reduces domain discrepancy in federated learning, enhancing knowledge transfer and outperforming existing personalized FL methods.

Findings

DBE significantly improves generalization and personalization in FL.

DBE outperforms ten state-of-the-art personalized FL methods.

Theoretical analysis confirms reduced domain discrepancy in representation space.

Abstract

Recently, federated learning (FL) is popular for its privacy-preserving and collaborative learning abilities. However, under statistically heterogeneous scenarios, we observe that biased data domains on clients cause a representation bias phenomenon and further degenerate generic representations during local training, i.e., the representation degeneration phenomenon. To address these issues, we propose a general framework Domain Bias Eliminator (DBE) for FL. Our theoretical analysis reveals that DBE can promote bi-directional knowledge transfer between server and client, as it reduces the domain discrepancy between server and client in representation space. Besides, extensive experiments on four datasets show that DBE can greatly improve existing FL methods in both generalization and personalization abilities. The DBE-equipped FL method can outperform ten state-of-the-art personalized FL methods by a large margin. Our code is public at https://github.com/TsingZ0/DBE.