FedDRL: A Trustworthy Federated Learning Model Fusion Method Based on Staged Reinforcement Learning

TL;DR

FedDRL introduces a staged reinforcement learning approach to improve federated learning by filtering malicious models and adaptively weighting trusted models, enhancing reliability without sacrificing accuracy.

Contribution

The paper presents a novel two-stage reinforcement learning method for model fusion in federated learning, addressing data heterogeneity and malicious model threats.

Findings

FedDRL outperforms baseline algorithms in reliability across five scenarios.

The method effectively filters malicious models and adaptively weights trusted models.

Maintains high accuracy comparable to traditional federated learning methods.

Abstract

Traditional federated learning uses the number of samples to calculate the weights of each client model and uses this fixed weight value to fusion the global model. However, in practical scenarios, each client's device and data heterogeneity leads to differences in the quality of each client's model. Thus the contribution to the global model is not wholly determined by the sample size. In addition, if clients intentionally upload low-quality or malicious models, using these models for aggregation will lead to a severe decrease in global model accuracy. Traditional federated learning algorithms do not address these issues. To solve this probelm, we propose FedDRL, a model fusion approach using reinforcement learning based on a two staged approach. In the first stage, Our method could filter out malicious models and selects trusted client models to participate in the model fusion. In the second stage, the FedDRL algorithm adaptively adjusts the weights of the trusted client models and aggregates the optimal global model. We also define five model fusion scenarios and compare our method with two baseline algorithms in those scenarios. The experimental results show that our algorithm has higher reliability than other algorithms while maintaining accuracy.