Constrained Pure Exploration Multi-Armed Bandits with a Fixed Budget
Fathima Zarin Faizal, Jayakrishnan Nair

TL;DR
This paper introduces a new algorithm for constrained pure exploration in multi-armed bandits with a fixed budget, optimizing a specific attribute while respecting constraints on others, with theoretical guarantees on its performance.
Contribution
It proposes extsc{Constrained-SR}, an algorithm based on Successive Rejects, with theoretical bounds and near-optimal decay rates for constrained pure exploration problems.
Findings
The algorithm achieves exponential decay of error probability with the budget.
It is nearly optimal in certain cases based on information theoretic bounds.
Provides a framework for risk-constrained optimization in bandit settings.
Abstract
We consider a constrained, pure exploration, stochastic multi-armed bandit formulation under a fixed budget. Each arm is associated with an unknown, possibly multi-dimensional distribution and is described by multiple attributes that are a function of this distribution. The aim is to optimize a particular attribute subject to user-defined constraints on the other attributes. This framework models applications such as financial portfolio optimization, where it is natural to perform risk-constrained maximization of mean return. We assume that the attributes can be estimated using samples from the arms' distributions and that these estimators satisfy suitable concentration inequalities. We propose an algorithm called \textsc{Constrained-SR} based on the Successive Rejects framework, which recommends an optimal arm and flags the instance as being feasible or infeasible. A key feature of…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAdvanced Bandit Algorithms Research · Machine Learning and Algorithms · Data Stream Mining Techniques
