Adversarial Attacks on Stochastic Bandits

TL;DR

This paper demonstrates how adversaries can manipulate reward signals in stochastic bandit algorithms like epsilon-greedy and UCB without prior knowledge, revealing security vulnerabilities in widely used decision-making systems.

Contribution

It introduces the first attack methods against popular bandit algorithms that do not require knowledge of mean rewards, with logarithmic effort proportional to problem difficulty.

Findings

Attacks can hijack bandit behavior to promote or obstruct actions.

Effort required for attack scales logarithmically with problem difficulty.

Vulnerabilities pose security threats in practical applications of bandits.

Abstract

We study adversarial attacks that manipulate the reward signals to control the actions chosen by a stochastic multi-armed bandit algorithm. We propose the first attack against two popular bandit algorithms: $\epsilon$-greedy and UCB, \emph{without} knowledge of the mean rewards. The attacker is able to spend only logarithmic effort, multiplied by a problem-specific parameter that becomes smaller as the bandit problem gets easier to attack. The result means the attacker can easily hijack the behavior of the bandit algorithm to promote or obstruct certain actions, say, a particular medical treatment. As bandits are seeing increasingly wide use in practice, our study exposes a significant security threat.