Contextual Decision-Making with Knapsacks Beyond the Worst Case

TL;DR

This paper advances the understanding of resource-constrained decision-making by providing algorithms with near-optimal regret bounds that surpass traditional worst-case guarantees, especially under specific problem structures.

Contribution

It introduces a new algorithm combining re-solving heuristics with distribution estimation that achieves logarithmic regret under certain conditions and extends results to continuous settings.

Findings

Achieves (1) regret when the fluid LP has a unique, non-degenerate solution.

Proves an unavoidable (( ext{T})) gap in worst-case scenarios.

Maintains near-(( ext{T})) regret in worst cases and under different feedback models.

Abstract

We study the framework of a dynamic decision-making scenario with resource constraints. In this framework, an agent, whose target is to maximize the total reward under the initial inventory, selects an action in each round upon observing a random request, leading to a reward and resource consumptions that are further associated with an unknown random external factor. While previous research has already established an $\widetilde{O}(\sqrt{T})$ worst-case regret for this problem, this work offers two results that go beyond the worst-case perspective: one for the worst-case gap between benchmarks and another for logarithmic regret rates. We first show that an $\Omega(\sqrt{T})$ distance between the commonly used fluid benchmark and the online optimum is unavoidable when the former has a degenerate optimal solution. On the algorithmic side, we merge the re-solving heuristic with distribution estimation skills and propose an algorithm that achieves an $\widetilde{O}(1)$ regret as long as the fluid LP has a unique and non-degenerate solution. Furthermore, we prove that our algorithm maintains a near-optimal $\widetilde{O}(\sqrt{T})$ regret even in the worst cases and extend these results to the setting where the request and external factor are continuous. Regarding information structure, our regret results are obtained under two feedback models, respectively, where the algorithm accesses the external factor at the end of each round and at the end of a round only when a non-null action is executed.