UCB Exploration for Fixed-Budget Bayesian Best Arm Identification

TL;DR

This paper introduces a Bayesian UCB-based algorithm for fixed-budget best-arm identification that learns prior information to improve efficiency, providing theoretical guarantees and outperforming existing methods in empirical tests.

Contribution

It proposes a novel UCB exploration algorithm that learns prior information, enhancing fixed-budget Bayesian best-arm identification with theoretical bounds and empirical superiority.

Findings

Achieves upper bounds on failure probability and simple regret of order $ ilde{O}(rac{ ext{sqrt}(K)}{n})$

Outperforms state-of-the-art baselines in empirical evaluations

Provides both theoretical analysis and practical improvements for Bayesian BAI

Abstract

We study best-arm identification (BAI) in the fixed-budget setting. Adaptive allocations based on upper confidence bounds (UCBs), such as UCBE, are known to work well in BAI. However, it is well-known that its optimal regret is theoretically dependent on instances, which we show to be an artifact in many fixed-budget BAI problems. In this paper we propose an UCB exploration algorithm that is both theoretically and empirically efficient for the fixed budget BAI problem under a Bayesian setting. The key idea is to learn prior information, which can enhance the performance of UCB-based BAI algorithm as it has done in the cumulative regret minimization problem. We establish bounds on the failure probability and the simple regret for the Bayesian BAI problem, providing upper bounds of order $\tilde{O}(\sqrt{K/n})$, up to logarithmic factors, where $n$ represents the budget and $K$ denotes the number of arms. Furthermore, we demonstrate through empirical results that our approach consistently outperforms state-of-the-art baselines.