Volumetric Spanners: an Efficient Exploration Basis for Learning

TL;DR

This paper introduces volumetric spanners, a new geometric exploration basis with low variance, enabling the first efficient and optimal regret algorithm for bandit linear optimization over general convex sets.

Contribution

It defines volumetric spanners and provides algorithms to construct them, extending regret minimization results to broader convex sets beyond special cases.

Findings

Efficient algorithms for constructing volumetric spanners.

First optimal regret algorithm for bandit linear optimization over general convex sets.

Extension of previous results to broader convex geometries.

Abstract

Numerous machine learning problems require an exploration basis - a mechanism to explore the action space. We define a novel geometric notion of exploration basis with low variance, called volumetric spanners, and give efficient algorithms to construct such a basis. We show how efficient volumetric spanners give rise to the first efficient and optimal regret algorithm for bandit linear optimization over general convex sets. Previously such results were known only for specific convex sets, or under special conditions such as the existence of an efficient self-concordant barrier for the underlying set.