FedSC: Federated Learning with Semantic-Aware Collaboration

TL;DR

FedSC introduces a semantic-aware federated learning approach that leverages class-relevant prototypes and contrastive learning to better handle data heterogeneity across clients, improving model convergence and performance.

Contribution

The paper proposes FedSC, a novel federated learning method that incorporates semantic-level prototypes and contrastive learning to effectively address data heterogeneity.

Findings

FedSC outperforms existing FL methods in heterogeneous data scenarios.

The use of relational and consistent prototypes enhances model convergence.

Experimental results validate the effectiveness and efficiency of FedSC.

Abstract

Federated learning (FL) aims to train models collaboratively across clients without sharing data for privacy-preserving. However, one major challenge is the data heterogeneity issue, which refers to the biased labeling preferences at multiple clients. A number of existing FL methods attempt to tackle data heterogeneity locally (e.g., regularizing local models) or globally (e.g., fine-tuning global model), often neglecting inherent semantic information contained in each client. To explore the possibility of using intra-client semantically meaningful knowledge in handling data heterogeneity, in this paper, we propose Federated Learning with Semantic-Aware Collaboration (FedSC) to capture client-specific and class-relevant knowledge across heterogeneous clients. The core idea of FedSC is to construct relational prototypes and consistent prototypes at semantic-level, aiming to provide fruitful class underlying knowledge and stable convergence signals in a prototype-wise collaborative way. On the one hand, FedSC introduces an inter-contrastive learning strategy to bring instance-level embeddings closer to relational prototypes with the same semantics and away from distinct classes. On the other hand, FedSC devises consistent prototypes via a discrepancy aggregation manner, as a regularization penalty to constrain the optimization region of the local model. Moreover, a theoretical analysis for FedSC is provided to ensure a convergence guarantee. Experimental results on various challenging scenarios demonstrate the effectiveness of FedSC and the efficiency of crucial components.