Shielding Federated Learning: Mitigating Byzantine Attacks with Less Constraints

TL;DR

This paper introduces Robust-FL, a novel federated learning scheme that effectively defends against Byzantine attacks without relying on common assumptions, by using historical models to detect malicious updates.

Contribution

Robust-FL is the first prediction-based federated learning method that does not depend on assumptions like fixed attack models or data distribution, enhancing robustness against diverse Byzantine attacks.

Findings

Achieves robustness without data or attack assumptions

Handles majority of malicious participants effectively

Generalizes to various Byzantine attack models

Abstract

Federated learning is a newly emerging distributed learning framework that facilitates the collaborative training of a shared global model among distributed participants with their privacy preserved. However, federated learning systems are vulnerable to Byzantine attacks from malicious participants, who can upload carefully crafted local model updates to degrade the quality of the global model and even leave a backdoor. While this problem has received significant attention recently, current defensive schemes heavily rely on various assumptions, such as a fixed Byzantine model, availability of participants' local data, minority attackers, IID data distribution, etc. To relax those constraints, this paper presents Robust-FL, the first prediction-based Byzantine-robust federated learning scheme where none of the assumptions is leveraged. The core idea of the Robust-FL is exploiting historical global model to construct an estimator based on which the local models will be filtered through similarity detection. We then cluster local models to adaptively adjust the acceptable differences between the local models and the estimator such that Byzantine users can be identified. Extensive experiments over different datasets show that our approach achieves the following advantages simultaneously: (i) independence of participants' local data, (ii) tolerance of majority attackers, (iii) generalization to variable Byzantine model.