Tractable Instances of Bilinear Maximization: Implementing LinUCB on Ellipsoids

TL;DR

This paper addresses the computational challenge of maximizing a bilinear form over convex sets and ellipsoids, introducing efficient algorithms for high-dimensional linear bandit problems with ellipsoidal action sets.

Contribution

The paper presents the first efficient algorithms for solving bilinear maximization over ellipsoids, enabling practical implementation of optimistic algorithms in high-dimensional linear bandits.

Findings

Efficient algorithms are possible for ellipsoidal sets.

No efficient algorithms exist for some convex sets unless P=NP.

First implementation of optimistic algorithms for high-dimensional linear bandits.

Abstract

We consider the maximization of $x^\top \theta$ over $(x,\theta) \in \mathcal{X} \times \Theta$, with $\mathcal{X} \subset \mathbb{R}^d$ convex and $\Theta \subset \mathbb{R}^d$ an ellipsoid. This problem is fundamental in linear bandits, as the learner must solve it at every time step using optimistic algorithms. We first show that for some sets $\mathcal{X}$ e.g. $\ell_p$ balls with $p>2$, no efficient algorithms exist unless $\mathcal{P} = \mathcal{NP}$. We then provide two novel algorithms solving this problem efficiently when $\mathcal{X}$ is a centered ellipsoid. Our findings provide the first known method to implement optimistic algorithms for linear bandits in high dimensions.