Bandits on graphs and structures

TL;DR

This thesis explores structured bandit problems on graphs and large action spaces, focusing on theoretical properties and practical algorithms for complex, high-dimensional decision-making scenarios.

Contribution

It provides a survey of the author's work on graph and structured bandits, addressing smoothness, side observations, influence maximization, and infinite action spaces.

Findings

Analyzed spectral bandits with reward smoothness assumptions.

Developed methods for influence maximization in graph bandit settings.

Covered kernel, polymatroid, and infinite-armed bandit models.

Abstract

The goal of this thesis is to investigate the structural properties of certain sequential problems in order to bring the solutions closer to a practical use. In the first part, we put a special emphasis on structures that can be represented as graphs on actions. In the second part, we study the large action spaces that can be of exponential size in the number of base actions or even infinite. For graph bandits, we consider the settings of smoothness of rewards (spectral bandits), side observations, and influence maximization. For large structured domains, we cover kernel bandits, polymatroid bandits, bandits for function optimization (including unknown smoothness), and infinitely many-arms bandits. The thesis aspires to be a survey of the author's contributions on graph and structured bandits.