Online Subset Selection using $\alpha$-Core with no Augmented Regret

TL;DR

This paper introduces SCore, an online subset selection policy based on the $ ext{ extalpha}$-Core concept, providing guarantees for a broad class of reward functions without augmented regret, and extending to optimistic learning scenarios.

Contribution

The paper proposes the SCore policy utilizing a new $ ext{ extalpha}$-Core characterization, offering efficient optimization for many reward functions and extending to optimistic learning with hints.

Findings

SCore achieves strong performance guarantees for submodular rewards.

The $ ext{ extalpha}$-Core characterization generalizes core concepts to online subset selection.

The policy extends to settings with additional untrusted reward hints.

Abstract

We revisit the classic problem of optimal subset selection in the online learning set-up. Assume that the set $[N]$ consists of $N$ distinct elements. On the $t$th round, an adversary chooses a monotone reward function $f_t: 2^{[N]} \to \mathbb{R}_+$ that assigns a non-negative reward to each subset of $[N].$ An online policy selects (perhaps randomly) a subset $S_t \subseteq [N]$ consisting of $k$ elements before the reward function $f_t$ for the $t$th round is revealed to the learner. As a consequence of its choice, the policy receives a reward of $f_t(S_t)$ on the $t$th round. Our goal is to design an online sequential subset selection policy to maximize the expected cumulative reward accumulated over a time horizon. In this connection, we propose an online learning policy called SCore (Subset Selection with Core) that solves the problem for a large class of reward functions. The proposed SCore policy is based on a new polyhedral characterization of the reward functions called $\alpha$-Core - a generalization of Core from the cooperative game theory literature. We establish a learning guarantee for the SCore policy in terms of a new performance metric called $\alpha$-augmented regret. In this new metric, the performance of the online policy is compared with an unrestricted offline benchmark that can select all $N$ elements at every round. We show that a large class of reward functions, including submodular, can be efficiently optimized with the SCore policy. We also extend the proposed policy to the optimistic learning set-up where the learner has access to additional untrusted hints regarding the reward functions. Finally, we conclude the paper with a list of open problems.