Stochastic $k$-Submodular Bandits with Full Bandit Feedback

TL;DR

This paper introduces the first sublinear regret bounds for online $k$-submodular bandit problems with full feedback, using offline-to-online transformation techniques for various function types and constraints.

Contribution

It develops online algorithms for multiple $k$-submodular bandit problems by transforming offline approximation algorithms, and analyzes their robustness in the online setting.

Findings

Achieved sublinear $eta$-regret bounds for different $k$-submodular bandit scenarios.

Extended offline-to-online framework to handle various constraints and function types.

Provided theoretical analysis of the robustness of offline algorithms in online contexts.

Abstract

In this paper, we present the first sublinear $\alpha$-regret bounds for online $k$-submodular optimization problems with full-bandit feedback, where $\alpha$ is a corresponding offline approximation ratio. Specifically, we propose online algorithms for multiple $k$-submodular stochastic combinatorial multi-armed bandit problems, including (i) monotone functions and individual size constraints, (ii) monotone functions with matroid constraints, (iii) non-monotone functions with matroid constraints, (iv) non-monotone functions without constraints, and (v) monotone functions without constraints. We transform approximation algorithms for offline $k$-submodular maximization problems into online algorithms through the offline-to-online framework proposed by Nie et al. (2023a). A key contribution of our work is analyzing the robustness of the offline algorithms.