Defending Against Sophisticated Poisoning Attacks with RL-based Aggregation in Federated Learning

TL;DR

This paper introduces AdaAggRL, an RL-based adaptive aggregation method that leverages data distribution stability to defend federated learning systems against advanced poisoning attacks, outperforming existing defenses.

Contribution

It proposes a novel adaptive aggregation approach using reinforcement learning and distribution similarity measures to enhance defense against sophisticated poisoning attacks in federated learning.

Findings

Significantly outperforms existing defenses on four datasets.

Benign clients show higher data distribution stability than malicious ones.

Adaptive aggregation effectively mitigates advanced poisoning attacks.

Abstract

Federated learning is highly susceptible to model poisoning attacks, especially those meticulously crafted for servers. Traditional defense methods mainly focus on updating assessments or robust aggregation against manually crafted myopic attacks. When facing advanced attacks, their defense stability is notably insufficient. Therefore, it is imperative to develop adaptive defenses against such advanced poisoning attacks. We find that benign clients exhibit significantly higher data distribution stability than malicious clients in federated learning in both CV and NLP tasks. Therefore, the malicious clients can be recognized by observing the stability of their data distribution. In this paper, we propose AdaAggRL, an RL-based Adaptive Aggregation method, to defend against sophisticated poisoning attacks. Specifically, we first utilize distribution learning to simulate the clients' data distributions. Then, we use the maximum mean discrepancy (MMD) to calculate the pairwise similarity of the current local model data distribution, its historical data distribution, and global model data distribution. Finally, we use policy learning to adaptively determine the aggregation weights based on the above similarities. Experiments on four real-world datasets demonstrate that the proposed defense model significantly outperforms widely adopted defense models for sophisticated attacks.