PDE-Based Optimal Strategy for Unconstrained Online Learning

TL;DR

This paper introduces a PDE-based framework for designing potential functions in unconstrained online linear optimization, resulting in a novel algorithm with optimal regret bounds that match theoretical lower bounds.

Contribution

It presents a PDE-driven method to generate potential functions, leading to the first algorithm achieving optimal regret bounds without the doubling trick in unconstrained online learning.

Findings

Developed a PDE-based framework for potential function design.

Produced a new algorithm with optimal regret bounds for 1-Lipschitz losses.

Proved the tightness of the regret bound with a matching lower bound.

Abstract

Unconstrained Online Linear Optimization (OLO) is a practical problem setting to study the training of machine learning models. Existing works proposed a number of potential-based algorithms, but in general the design of these potential functions relies heavily on guessing. To streamline this workflow, we present a framework that generates new potential functions by solving a Partial Differential Equation (PDE). Specifically, when losses are 1-Lipschitz, our framework produces a novel algorithm with anytime regret bound $C\sqrt{T}+||u||\sqrt{2T}[\sqrt{\log(1+||u||/C)}+2]$, where $C$ is a user-specified constant and $u$ is any comparator unknown and unbounded a priori. Such a bound attains an optimal loss-regret trade-off without the impractical doubling trick. Moreover, a matching lower bound shows that the leading order term, including the constant multiplier $\sqrt{2}$, is tight. To our knowledge, the proposed algorithm is the first to achieve such optimalities.