Approximate optimality and the risk/reward tradeoff in a class of bandit problems

TL;DR

This paper investigates the conditions under which a decision-maker should specialize or diversify in a class of bandit problems with known payoff distributions, emphasizing the role of risk attitudes and using a new central limit theorem.

Contribution

It introduces a framework for understanding approximate optimality in large-horizon bandit problems, highlighting when specialization or diversification is optimal based on risk attitude.

Findings

Specialization is asymptotically optimal under constant risk attitudes.

Diversification becomes optimal when risk attitudes vary.

A new central limit theorem underpins the analysis.

Abstract

This paper studies a sequential decision problem where payoff distributions are known and where the riskiness of payoffs matters. Equivalently, it studies sequential choice from a repeated set of independent lotteries. The decision-maker is assumed to pursue strategies that are approximately optimal for large horizons. By exploiting the tractability afforded by asymptotics, conditions are derived characterizing when specialization in one action or lottery throughout is asymptotically optimal and when optimality requires intertemporal diversification. The key is the constancy or variability of risk attitude. The main technical tool is a new central limit theorem.