A Huber Loss Minimization Approach to Byzantine Robust Federated Learning

TL;DR

This paper proposes a Huber loss-based aggregator for federated learning that enhances robustness against adversarial attacks, offering theoretical guarantees and flexibility for different data distributions and client data sizes.

Contribution

It introduces a novel Huber loss minimization method for robust federated learning, with theoretical analysis showing advantages over existing approaches under various data assumptions.

Findings

Optimal dependence on attack ratio $eta$

No need for precise attack ratio knowledge

Supports clients with unequal data sizes

Abstract

Federated learning systems are susceptible to adversarial attacks. To combat this, we introduce a novel aggregator based on Huber loss minimization, and provide a comprehensive theoretical analysis. Under independent and identically distributed (i.i.d) assumption, our approach has several advantages compared to existing methods. Firstly, it has optimal dependence on $\epsilon$, which stands for the ratio of attacked clients. Secondly, our approach does not need precise knowledge of $\epsilon$. Thirdly, it allows different clients to have unequal data sizes. We then broaden our analysis to include non-i.i.d data, such that clients have slightly different distributions.