Towards Performance-Enhanced Model-Contrastive Federated Learning using Historical Information in Heterogeneous Scenarios

TL;DR

This paper introduces PMFL, a federated learning framework that leverages historical information on both client and server sides to enhance performance in heterogeneous environments with varying data distributions and participation frequencies.

Contribution

The paper proposes a novel PMFL framework that integrates historical local and global models, and adaptive weighting based on participation counts, to improve federated learning in heterogeneous scenarios.

Findings

PMFL outperforms existing FL methods in heterogeneous settings.

Incorporating historical models stabilizes performance across rounds.

Adaptive weighting reduces bias caused by participation frequency.

Abstract

Federated Learning (FL) enables multiple nodes to collaboratively train a model without sharing raw data. However, FL systems are usually deployed in heterogeneous scenarios, where nodes differ in both data distributions and participation frequencies, which undermines the FL performance. To tackle the above issue, this paper proposes PMFL, a performance-enhanced model-contrastive federated learning framework using historical training information. Specifically, on the node side, we design a novel model-contrastive term into the node optimization objective by incorporating historical local models to capture stable contrastive points, thereby improving the consistency of model updates in heterogeneous data distributions. On the server side, we utilize the cumulative participation count of each node to adaptively adjust its aggregation weight, thereby correcting the bias in the global objective caused by different node participation frequencies. Furthermore, the updated global model incorporates historical global models to reduce its fluctuations in performance between adjacent rounds. Extensive experiments demonstrate that PMFL achieves superior performance compared with existing FL methods in heterogeneous scenarios.