Non-Stationary Online Resource Allocation: Learning from a Single Sample

TL;DR

This paper introduces a novel online resource allocation method that effectively handles arbitrary non-stationarity with minimal offline data, achieving near-optimal regret bounds in complex, dynamic environments.

Contribution

It proposes a type-dependent quantile-based meta-policy that operates with only one sample per period, handling non-stationarity without variation constraints, and achieves the first poly-logarithmic regret guarantee.

Findings

Static threshold policy attains $ ilde{O}( oot{T} ull)$ regret with reward-observed samples.

Partially adaptive policy achieves $ ilde{O}(T)$ regret under minimal-arrival assumptions.

Fully adaptive policy attains $O(( ext{log } T)^3)$ regret in non-stationary multi-resource allocation.

Abstract

We study online resource allocation under non-stationary demand with a minimum offline data requirement. In this problem, a decision-maker must allocate multiple types of resources to sequentially arriving queries over a finite horizon. Each query belongs to a finite set of types with fixed resource consumption and a stochastic reward drawn from an unknown, type-specific distribution. Critically, the environment exhibits arbitrary non-stationarity -- arrival distributions may shift unpredictably-while the algorithm requires only one historical sample per period to operate effectively. We distinguish two settings based on sample informativeness: (i) reward-observed samples containing both query type and reward realization, and (ii) the more challenging type-only samples revealing only query type information. We propose a novel type-dependent quantile-based meta-policy that decouples the problem into modular components: reward distribution estimation, optimization of target service probabilities via fluid relaxation, and real-time decisions through dynamic acceptance thresholds. For reward-observed samples, our static threshold policy achieves $\tilde{O}(\sqrt{T})$ regret. For type-only samples, we first establish that sublinear regret is impossible without additional structure; under a mild minimum-arrival-probability assumption, we design both a partially adaptive policy attaining the same $\tilde{O}({T})$ bound and, more significantly, a fully adaptive resolving policy with careful rounding that achieves the first poly-logarithmic regret guarantee of $O((\log T)^3)$ for non-stationary multi-resource allocation. Our framework advances prior work by operating with minimal offline data (one sample per period), handling arbitrary non-stationarity without variation-budget assumptions, and supporting multiple resource constraints.