Dual Averaging on Compactly-Supported Distributions And Application to No-Regret Learning on a Continuum

TL;DR

This paper develops a dual averaging method for online learning over a continuum of decisions, providing regret bounds and demonstrating sublinear regret on certain non-convex sets.

Contribution

It introduces a dual averaging algorithm with $ ext{omega}$-potentials for continuum decision spaces and proves regret bounds under weaker conditions than convexity.

Findings

Achieves sublinear regret on uniformly fat sets.

Provides regret bounds for dual averaging on $L^2(S)$.

Extends online convex optimization to continuum decision spaces.

Abstract

We consider an online learning problem on a continuum. A decision maker is given a compact feasible set $S$, and is faced with the following sequential problem: at iteration~$t$, the decision maker chooses a distribution $x^{(t)} \in \Delta(S)$, then a loss function $\ell^{(t)} : S \to \mathbb{R}_+$ is revealed, and the decision maker incurs expected loss $\langle \ell^{(t)}, x^{(t)} \rangle = \mathbb{E}_{s \sim x^{(t)}} \ell^{(t)}(s)$. We view the problem as an online convex optimization problem on the space $\Delta(S)$ of Lebesgue-continnuous distributions on $S$. We prove a general regret bound for the Dual Averaging method on $L^2(S)$, then prove that dual averaging with $\omega$-potentials (a class of strongly convex regularizers) achieves sublinear regret when $S$ is uniformly fat (a condition weaker than convexity).