Robust Decentralized Multi-armed Bandits: From Corruption-Resilience to Byzantine-Resilience

TL;DR

This paper introduces DeMABAR, a robust decentralized multi-armed bandit algorithm resilient to adversarial corruption and Byzantine attacks, ensuring low regret and robustness in multi-agent collaborative settings.

Contribution

The paper develops a novel algorithm, DeMABAR, that provides resilience against both reward corruption and Byzantine agent attacks in decentralized multi-armed bandits.

Findings

DeMABAR achieves regret bounds proportional to corruption budget.

The algorithm is effective against Byzantine agents with arbitrary behaviors.

Numerical experiments confirm robustness and practical effectiveness.

Abstract

Decentralized cooperative multi-agent multi-armed bandits (DeCMA2B) considers how multiple agents collaborate in a decentralized multi-armed bandit setting. Though this problem has been extensively studied in previous work, most existing methods remain susceptible to various adversarial attacks. In this paper, we first study DeCMA2B with adversarial corruption, where an adversary can corrupt reward observations of all agents with a limited corruption budget. We propose a robust algorithm, called DeMABAR, which ensures that each agent's individual regret suffers only an additive term proportional to the corruption budget. Then we consider a more realistic scenario where the adversary can only attack a small number of agents. Our theoretical analysis shows that the DeMABAR algorithm can also almost completely eliminate the influence of adversarial attacks and is inherently robust in the Byzantine setting, where an unknown fraction of the agents can be Byzantine, i.e., may arbitrarily select arms and communicate wrong information. We also conduct numerical experiments to illustrate the robustness and effectiveness of the proposed method.