FedCoSR: Personalized Federated Learning with Contrastive Shareable Representations for Label Heterogeneity in Non-IID Data

TL;DR

FedCoSR is a personalized federated learning method that uses contrastive learning and adaptive aggregation to improve accuracy and fairness across clients with label distribution skew and data scarcity.

Contribution

The paper introduces FedCoSR, a novel federated learning algorithm that combines contrastive shareable representations with adaptive local aggregation for label heterogeneity.

Findings

Improves accuracy on non-IID datasets with label skew.

Enhances fairness among clients with scarce data.

Outperforms existing federated learning methods in experiments.

Abstract

Heterogeneity arising from label distribution skew and data scarcity can cause inaccuracy and unfairness in intelligent communication applications that heavily rely on distributed computing. To deal with it, this paper proposes a novel personalized federated learning algorithm, named Federated Contrastive Shareable Representations (FedCoSR), to facilitate knowledge sharing among clients while maintaining data privacy. Specifically, the parameters of local models' shallow layers and typical local representations are both considered as shareable information for the server and are aggregated globally. To address performance degradation caused by label distribution skew among clients, contrastive learning is adopted between local and global representations to enrich local knowledge. Additionally, to ensure fairness for clients with scarce data, FedCoSR introduces adaptive local aggregation to coordinate the global model involvement in each client. Our simulations demonstrate FedCoSR's effectiveness in mitigating label heterogeneity by achieving accuracy and fairness improvements over existing methods on datasets with varying degrees of label heterogeneity.