Bandits with Dynamic Arm-acquisition Costs
Anand Kalvit, Assaf Zeevi

TL;DR
This paper studies a bandit problem where adding new arms incurs costs and affects the probability of finding optimal arms, proposing an adaptive algorithm with optimal long-term performance under certain conditions.
Contribution
It introduces a novel bandit model with dynamic arm acquisition costs and characterizes conditions for achieving sub-linear regret, along with a near-optimal adaptive algorithm.
Findings
Necessary condition for sub-linear regret identified
UCB-inspired algorithm shown to be long-run optimal
Conditions for achievability of optimal performance established
Abstract
We consider a bandit problem where at any time, the decision maker can add new arms to her consideration set. A new arm is queried at a cost from an "arm-reservoir" containing finitely many "arm-types," each characterized by a distinct mean reward. The cost of query reflects in a diminishing probability of the returned arm being optimal, unbeknown to the decision maker; this feature encapsulates defining characteristics of a broad class of operations-inspired online learning problems, e.g., those arising in markets with churn, or those involving allocations subject to costly resource acquisition. The decision maker's goal is to maximize her cumulative expected payoffs over a sequence of n pulls, oblivious to the statistical properties as well as types of the queried arms. We study two natural modes of endogeneity in the reservoir distribution, and characterize a necessary condition for…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAdvanced Bandit Algorithms Research · Auction Theory and Applications · Data Stream Mining Techniques
MethodsMulti-Head Attention · Attention Is All You Need · Softmax · Linear Layer · InfoNCE · Contrastive Predictive Coding · Dense Connections · Relative Position Encodings · Position-Wise Feed-Forward Layer · Global-Local Attention
