Online Algorithms for Repeated Optimal Stopping: Balancing Baseline Guarantees and Regret

TL;DR

This paper investigates the feasibility of achieving both strong per-round performance guarantees and sublinear regret in repeated optimal stopping problems, providing theoretical insights and a general algorithmic framework.

Contribution

It characterizes when and how these objectives can be simultaneously achieved, and introduces a framework that attains both guarantees in canonical problems.

Findings

Maintaining per-round guarantees under semi-bandit feedback incurs linear regret.

Requiring per-round guarantees in every round is incompatible with sublinear regret even with full feedback.

The proposed framework achieves high-probability per-round guarantees and sublinear regret in canonical problems.

Abstract

We study the repeated optimal stopping problem, in which the same optimal stopping instance with an unknown distribution is solved repeatedly over $T$ rounds. We aim to simultaneously achieve strong per-round performance guarantees relative to a given baseline and sublinear regret across all rounds. Our primary contribution is a comprehensive theoretical characterization of whether and when these two objectives are compatible. First, under standard semi-bandit feedback, we prove that maintaining the per-round guarantee forces regret of $\Omega(T / \log T)$. Second, even under full feedback, we show that requiring almost-sure satisfaction of the per-round guarantee in every round is incompatible with sublinear regret. Third, under full feedback, we propose a general algorithmic framework that achieves both sublinear regret and the per-round guarantee with high probability. Our framework applies to canonical problems, including the prophet inequality, the secretary problem, and their variants under adversarial, random, and i.i.d. input models. For example, in the repeated prophet inequality problem, our method guarantees that, with high probability in each round, its expected reward is at least that of the classical single-sample algorithm, which achieves a $1/2$ competitive ratio, while simultaneously ensuring $\tilde{O}(\sqrt{T})$ regret. Furthermore, we establish a regret lower bound of $\Omega(\sqrt{T})$ even in the i.i.d. model, which is nearly tight with respect to the number of rounds.