A Novel Pearson Correlation-Based Merging Algorithm for Robust Distributed Machine Learning with Heterogeneous Data

TL;DR

This paper introduces a Pearson correlation-based merging algorithm for federated learning that enhances robustness and reduces communication overhead in heterogeneous data scenarios.

Contribution

It proposes a novel merging method using Pearson correlation to form intermediary nodes, improving model robustness and efficiency in federated learning with heterogeneous data.

Findings

Achieved higher accuracy under normal and adverse conditions

Reduced communication overhead by merging similar local models

Enhanced robustness against data poisoning attacks

Abstract

Federated learning faces significant challenges in scenarios with heterogeneous data distributions and adverse network conditions, such as delays, packet loss, and data poisoning attacks. This paper proposes a novel method based on the SCAFFOLD algorithm to improve the quality of local updates and enhance the robustness of the global model. The key idea is to form intermediary nodes by merging local models with high similarity, using the Pearson correlation coefficient as a similarity measure. The proposed merging algorithm reduces the number of local nodes while maintaining the accuracy of the global model, effectively addressing communication overhead and bandwidth consumption. Experimental results on the MNIST dataset under simulated federated learning scenarios demonstrate the method's effectiveness. After 10 rounds of training using a CNN model, the proposed approach achieved accuracies of 0.82, 0.73, and 0.66 under normal conditions, packet loss and data poisoning attacks, respectively, outperforming the baseline SCAFFOLD algorithm. These results highlight the potential of the proposed method to improve efficiency and resilience in federated learning systems.