A Unified Analysis Method for Online Optimization in Normed Vector Space

TL;DR

This paper presents a unified theoretical framework for online optimization in normed vector spaces, generalizing existing methods and achieving tighter regret bounds through novel analysis techniques.

Contribution

It introduces a unified analysis method for optimistic algorithms, extends online convex optimization to monotone operators, and improves regret bounds with the concept of $\

Findings

Regret bounds are the tightest possible due to $\

Regret bounds of normalized exponentiated subgradient and greedy/lazy projection surpass previous results.

Unified analysis applies to both Optimistic-DA and Optimistic-MD in normed vector spaces.

Abstract

This paper studies online optimization from a high-level unified theoretical perspective. We not only generalize both Optimistic-DA and Optimistic-MD in normed vector space, but also unify their analysis methods for dynamic regret. Regret bounds are the tightest possible due to the introduction of $\phi$-convex. As instantiations, regret bounds of normalized exponentiated subgradient and greedy/lazy projection are better than the currently known optimal results. By replacing losses of online game with monotone operators, and extending the definition of regret, namely regret$^n$, we extend online convex optimization to online monotone optimization.