Functional Bandits

TL;DR

This paper introduces the functional bandit problem, focusing on optimizing known functionals of unknown reward distributions, and proposes an efficient method combining estimation and elimination with theoretical guarantees.

Contribution

It presents a novel approach for functional bandits that effectively handles complex functionals, expanding the applicability of bandit algorithms to new domains.

Findings

Achieves provably efficient performance guarantees.

Effectively handles risk management and information theory functionals.

Demonstrates improved arm identification in complex settings.

Abstract

We introduce the functional bandit problem, where the objective is to find an arm that optimises a known functional of the unknown arm-reward distributions. These problems arise in many settings such as maximum entropy methods in natural language processing, and risk-averse decision-making, but current best-arm identification techniques fail in these domains. We propose a new approach, that combines functional estimation and arm elimination, to tackle this problem. This method achieves provably efficient performance guarantees. In addition, we illustrate this method on a number of important functionals in risk management and information theory, and refine our generic theoretical results in those cases.