Linear Bandits beyond Inner Product Spaces, the case of Bandit Optimal Transport

TL;DR

This paper extends linear bandit theory beyond inner product spaces by incorporating Optimal Transport problems, proposing a new algorithm that achieves competitive regret bounds without relying on traditional inner product assumptions.

Contribution

It introduces a refined OFUL algorithm embedding actions into a Hilbertian subspace, enabling efficient learning in non-inner product structured problems like Optimal Transport.

Findings

Achieves regret bounds similar to classical OFUL up to an approximation term.

Interpolates regret between (T) and (T), depending on cost regularity.

Recovers parametric rate ( ext{d}T) in finite-dimensional cases.

Abstract

Linear bandits have long been a central topic in online learning, with applications ranging from recommendation systems to adaptive clinical trials. Their general learnability has been established when the objective is to minimise the inner product between a cost parameter and the decision variable. While this is highly general, this reliance on an inner product structure belies the name of \emph{linear} bandits, and fails to account for problems such as Optimal Transport. Using the Kantorovich formulation of Optimal Transport as an example, we show that an inner product structure is \emph{not} necessary to achieve efficient learning in linear bandits. We propose a refinement of the classical OFUL algorithm that operates by embedding the action set into a Hilbertian subspace, where confidence sets can be built via least-squares estimation. Actions are then constrained to this subspace by penalising optimism. The analysis is completed by leveraging convergence results from penalised (entropic) transport to the Kantorovich problem. Up to this approximation term, the resulting algorithm achieves the same trajectorial regret upper bounds as the OFUL algorithm, which we turn into worst-case regret using functional regression techniques. Its regret interpolates between $\tilde{\mathcal O}(\sqrt{T})$ and ${\mathcal O}(T)$, depending on the regularity of the cost function, and recovers the parametric rate $\tilde{\mathcal O}(\sqrt{dT})$ in finite-dimensional settings.