No-Regret Learning Under Adversarial Resource Constraints: A Spending Plan Is All You Need!

TL;DR

This paper introduces a framework for online decision making under adversarial resource constraints using spending plans, achieving sublinear regret with novel primal-dual algorithms that adapt to budget distributions.

Contribution

It proposes a spending plan-guided approach with primal-dual algorithms for resource-constrained online learning, addressing adversarial changes and imbalanced budgets.

Findings

Algorithms achieve sublinear regret relative to spending plan baselines.

Performance improves with well-balanced budget distribution.

Robust variants handle highly imbalanced spending plans.

Abstract

We study online decision making problems under resource constraints, where both reward and cost functions are drawn from distributions that may change adversarially over time. We focus on two canonical settings: $(i)$ online resource allocation where rewards and costs are observed before action selection, and $(ii)$ online learning with resource constraints where they are observed after action selection, under full feedback or bandit feedback. It is well known that achieving sublinear regret in these settings is impossible when reward and cost distributions may change arbitrarily over time. To address this challenge, we analyze a framework in which the learner is guided by a spending plan--a sequence prescribing expected resource usage across rounds. We design general (primal-)dual methods that achieve sublinear regret with respect to baselines that follow the spending plan. Crucially, the performance of our algorithms improves when the spending plan ensures a well-balanced distribution of the budget across rounds. We additionally provide a robust variant of our methods to handle worst-case scenarios where the spending plan is highly imbalanced. To conclude, we study the regret of our algorithms when competing against benchmarks that deviate from the prescribed spending plan.