Bandit Social Learning: Exploration under Myopic Behavior

TL;DR

This paper analyzes social learning in multi-armed bandit settings with myopic agents and behavioral biases, revealing fundamental failures of greedy algorithms and proposing conditions for successful exploration.

Contribution

It provides the first comprehensive theoretical analysis of exploration failures under behavioral biases in social bandit learning, and identifies conditions for effective exploration strategies.

Findings

Greedy bandit algorithms fail under various behavioral biases.

A small fraction of optimistic agents can achieve near-optimal regret.

Pessimism does not prevent learning failures.

Abstract

We study social learning dynamics motivated by reviews on online platforms. The agents collectively follow a simple multi-armed bandit protocol, but each agent acts myopically, without regards to exploration. We allow the greedy (exploitation-only) algorithm, as well as a wide range of behavioral biases. Specifically, we allow myopic behaviors that are consistent with (parameterized) confidence intervals for the arms' expected rewards. We derive stark learning failures for any such behavior, and provide matching positive results. The learning-failure results extend to Bayesian agents and Bayesian bandit environments. In particular, we obtain general, quantitatively strong results on failure of the greedy bandit algorithm, both for ``frequentist" and ``Bayesian" versions. Failure results known previously are quantitatively weak, and either trivial or very specialized. Thus, we provide a theoretical foundation for designing non-trivial bandit algorithms, \ie algorithms that intentionally explore, which has been missing from the literature. Our general behavioral model can be interpreted as agents' optimism or pessimism. The matching positive results entail a maximal allowed amount of optimism. Moreover, we find that no amount of pessimism helps against the learning failures, whereas even a small-but-constant fraction of extreme optimists avoids the failures and leads to near-optimal regret rates.