BOBA: Byzantine-Robust Federated Learning with Label Skewness

TL;DR

BOBA is a novel federated learning method designed to be robust against Byzantine attacks and label skewness in non-IID data settings, ensuring unbiasedness and convergence.

Contribution

We introduce BOBA, a two-stage algorithm that improves robustness and unbiasedness in non-IID federated learning with label skewness, with proven convergence.

Findings

BOBA outperforms baselines in robustness across datasets.

BOBA maintains unbiasedness in label skewed settings.

BOBA converges with optimal error bounds.

Abstract

In federated learning, most existing robust aggregation rules (AGRs) combat Byzantine attacks in the IID setting, where client data is assumed to be independent and identically distributed. In this paper, we address label skewness, a more realistic and challenging non-IID setting, where each client only has access to a few classes of data. In this setting, state-of-the-art AGRs suffer from selection bias, leading to significant performance drop for particular classes; they are also more vulnerable to Byzantine attacks due to the increased variation among gradients of honest clients. To address these limitations, we propose an efficient two-stage method named BOBA. Theoretically, we prove the convergence of BOBA with an error of the optimal order. Our empirical evaluations demonstrate BOBA's superior unbiasedness and robustness across diverse models and datasets when compared to various baselines. Our code is available at https://github.com/baowenxuan/BOBA .