Byzantine-robust Federated Learning through Collaborative Malicious Gradient Filtering

TL;DR

This paper introduces SignGuard, a novel method for federated learning that detects and filters malicious gradients without auxiliary data, using gradient sign analysis to improve robustness against sophisticated model poisoning attacks.

Contribution

We propose SignGuard, a new collaborative filtering approach leveraging gradient sign information to defend against Byzantine attacks without relying on auxiliary data.

Findings

SignGuard effectively detects malicious gradients in federated learning.

The method outperforms existing defenses against advanced poisoning attacks.

Experimental results show improved accuracy and robustness in image and text classification tasks.

Abstract

Gradient-based training in federated learning is known to be vulnerable to faulty/malicious clients, which are often modeled as Byzantine clients. To this end, previous work either makes use of auxiliary data at parameter server to verify the received gradients (e.g., by computing validation error rate) or leverages statistic-based methods (e.g. median and Krum) to identify and remove malicious gradients from Byzantine clients. In this paper, we remark that auxiliary data may not always be available in practice and focus on the statistic-based approach. However, recent work on model poisoning attacks has shown that well-crafted attacks can circumvent most of median- and distance-based statistical defense methods, making malicious gradients indistinguishable from honest ones. To tackle this challenge, we show that the element-wise sign of gradient vector can provide valuable insight in detecting model poisoning attacks. Based on our theoretical analysis of the \textit{Little is Enough} attack, we propose a novel approach called \textit{SignGuard} to enable Byzantine-robust federated learning through collaborative malicious gradient filtering. More precisely, the received gradients are first processed to generate relevant magnitude, sign, and similarity statistics, which are then collaboratively utilized by multiple filters to eliminate malicious gradients before final aggregation. Finally, extensive experiments of image and text classification tasks are conducted under recently proposed attacks and defense strategies. The numerical results demonstrate the effectiveness and superiority of our proposed approach. The code is available at \textit{\url{https://github.com/JianXu95/SignGuard}}