Byzantine-Robust Learning on Heterogeneous Data via Gradient Splitting

TL;DR

This paper introduces GAS, a method that enhances Byzantine-robust federated learning in non-IID data environments by addressing gradient heterogeneity and dimensionality issues, improving convergence and robustness.

Contribution

We propose GAS, a novel approach that adapts existing robust aggregation rules to non-IID data, with theoretical convergence analysis and empirical validation.

Findings

GAS improves robustness of federated learning under Byzantine attacks in non-IID settings.

Experimental results show GAS outperforms existing methods on real-world datasets.

Theoretical analysis confirms convergence of GAS with robust aggregation rules.

Abstract

Federated learning has exhibited vulnerabilities to Byzantine attacks, where the Byzantine attackers can send arbitrary gradients to a central server to destroy the convergence and performance of the global model. A wealth of robust AGgregation Rules (AGRs) have been proposed to defend against Byzantine attacks. However, Byzantine clients can still circumvent robust AGRs when data is non-Identically and Independently Distributed (non-IID). In this paper, we first reveal the root causes of performance degradation of current robust AGRs in non-IID settings: the curse of dimensionality and gradient heterogeneity. In order to address this issue, we propose GAS, a \shorten approach that can successfully adapt existing robust AGRs to non-IID settings. We also provide a detailed convergence analysis when the existing robust AGRs are combined with GAS. Experiments on various real-world datasets verify the efficacy of our proposed GAS. The implementation code is provided in https://github.com/YuchenLiu-a/byzantine-gas.