Learning with a Budget: Identifying the Best Arm with Resource Constraints

TL;DR

This paper introduces a resource-aware algorithm for identifying the best option among alternatives when resources are limited, unifying analysis for stochastic and deterministic resource consumption.

Contribution

It proposes the SH-RR algorithm that incorporates resource constraints into best arm identification, extending classical methods to resource-aware settings.

Findings

The SH-RR algorithm effectively balances resource usage and identification accuracy.

Theoretical analysis covers both stochastic and deterministic resource consumption.

Experimental results demonstrate improved efficiency over existing methods.

Abstract

In many applications, evaluating the effectiveness of different alternatives comes with varying costs or resource usage. Motivated by such heterogeneity, we study the Best Arm Identification with Resource Constraints (BAIwRC) problem, where an agent seeks to identify the best alternative (aka arm) in the presence of resource constraints. Each arm pull consumes one or more types of limited resources. We make two key contributions. First, we propose the Successive Halving with Resource Rationing (SH-RR) algorithm, which integrates resource-aware allocation into the classical successive halving framework on best arm identification. The SH-RR algorithm unifies the theoretical analysis for both the stochastic and deterministic consumption settings, with a new \textit{effective consumption measure