Federated Multi-Armed Bandits Under Byzantine Attacks

TL;DR

This paper introduces Fed-MoM-UCB, a robust federated multi-armed bandit algorithm that effectively handles Byzantine clients, ensuring low regret and communication efficiency in adversarial settings.

Contribution

It proposes a median-of-means based algorithm for federated MAB that is resilient to Byzantine attacks and analyzes its regret and communication efficiency.

Findings

Regret is bounded when Byzantine clients are less than half the cohort.

Fed-MoM-UCB outperforms baselines under Byzantine attacks.

The algorithm balances exploration, exploitation, and robustness.

Abstract

Multi-armed bandits (MAB) is a sequential decision-making model in which the learner controls the trade-off between exploration and exploitation to maximize its cumulative reward. Federated multi-armed bandits (FMAB) is an emerging framework where a cohort of learners with heterogeneous local models play an MAB game and communicate their aggregated feedback to a server to learn a globally optimal arm. Two key hurdles in FMAB are communication-efficient learning and resilience to adversarial attacks. To address these issues, we study the FMAB problem in the presence of Byzantine clients who can send false model updates threatening the learning process. We analyze the sample complexity and the regret of $\beta$-optimal arm identification. We borrow tools from robust statistics and propose a median-of-means (MoM)-based online algorithm, Fed-MoM-UCB, to cope with Byzantine clients. In particular, we show that if the Byzantine clients constitute less than half of the cohort, the cumulative regret with respect to $\beta$-optimal arms is bounded over time with high probability, showcasing both communication efficiency and Byzantine resilience. We analyze the interplay between the algorithm parameters, a discernibility margin, regret, communication cost, and the arms' suboptimality gaps. We demonstrate Fed-MoM-UCB's effectiveness against the baselines in the presence of Byzantine attacks via experiments.