Projection-Free Online Optimization with Stochastic Gradient: From Convexity to Submodularity

TL;DR

This paper introduces the first online projection-free algorithms using stochastic gradients for convex and submodular optimization, achieving optimal regret bounds and enabling efficient large-scale online learning.

Contribution

It presents Meta-Frank-Wolfe and One-Shot Frank-Wolfe algorithms that extend online optimization to non-convex submodular functions without projections, with theoretical regret guarantees.

Findings

Meta-Frank-Wolfe achieves $O(\sqrt{T})$ regret bounds.

One-Shot Frank-Wolfe achieves $O(T^{2/3})$ regret bounds.

Algorithms outperform existing methods in experiments.

Abstract

Online optimization has been a successful framework for solving large-scale problems under computational constraints and partial information. Current methods for online convex optimization require either a projection or exact gradient computation at each step, both of which can be prohibitively expensive for large-scale applications. At the same time, there is a growing trend of non-convex optimization in machine learning community and a need for online methods. Continuous DR-submodular functions, which exhibit a natural diminishing returns condition, have recently been proposed as a broad class of non-convex functions which may be efficiently optimized. Although online methods have been introduced, they suffer from similar problems. In this work, we propose Meta-Frank-Wolfe, the first online projection-free algorithm that uses stochastic gradient estimates. The algorithm relies on a careful sampling of gradients in each round and achieves the optimal $O( \sqrt{T})$ adversarial regret bounds for convex and continuous submodular optimization. We also propose One-Shot Frank-Wolfe, a simpler algorithm which requires only a single stochastic gradient estimate in each round and achieves an $O(T^{2/3})$ stochastic regret bound for convex and continuous submodular optimization. We apply our methods to develop a novel "lifting" framework for the online discrete submodular maximization and also see that they outperform current state-of-the-art techniques on various experiments.

Equations

$$f(A)+f(B)\ge f(A\cup B)+f(A\cap B)$$

$$f(\mathbf{x})+f(\mathbf{y})\ge f(\mathbf{x}\vee \mathbf{y})+f(\mathbf{x}\wedge \mathbf{y})$$

$$\nabla f(\mathbf{x})\ge \nabla f(\mathbf{y})$$

$$f(\mathbf{y})\le f(\mathbf{x})+⟨\nabla f(\mathbf{x}),\mathbf{y}-\mathbf{x}⟩$$

$${\mathcal{R}}_T\triangleq \sum _{t=1}^T{f}_t({\mathbf{x}}_t)-\underset{\mathbf{x}\in \mathcal{K}}{\inf }\sum _{t=1}^T{f}_t(\mathbf{x})$$

$${\mathcal{S}\mathcal{R}}_T\triangleq \sum _{t=1}^{T}f({\mathbf{x}}_t)-T\cdot \underset{\mathbf{x}\in \mathcal{K}}{\inf }f(\mathbf{x})$$

$$\alpha \text{mbox}-{\mathcal{R}}_T\triangleq \alpha \underset{\mathbf{x}\in \mathcal{K}}{\sup }\sum _{t=1}^T{f}_t(\mathbf{x})-\sum _{t=1}^T{f}_t({\mathbf{x}}_t)$$

$${\mathbf{x}}_t^{(k+1)}\leftarrow (1-{\eta }_k){\mathbf{x}}_t^{(k)}+{\eta }_k{\mathbf{v}}_t^{(k)}$$

$${\mathbf{x}}_t^{(k+1)}\leftarrow {\mathbf{x}}_t^{(k)}+{\eta }_k{\mathbf{v}}_t^{(k)}$$

$$\frac{4TD{Q}^{1/2}}{K^{1/3}}+\frac{4T}{K}\left(M+\frac{L{D}^2}{3}\log (K+1)\right)+\frac{4}{3}{\mathcal{R}}_T^{\mathcal{E}}$$

$$\frac{3TD{Q}^{1/2}}{2K^{1/3}}+\frac{L{D}^{2}T}{2K}+{\mathcal{R}}_T^{\mathcal{E}}$$

$$4M\log (T+1)+6Q^{1/2}D{T}^{2/3}+\frac{4}{3}L{D}^2{\log }^2(T+3)$$

$$(1-1/e)M+\frac{3D{Q}^{1/2}}{10}(3T^{2/3}+2T^{-1})+\frac{L{D}^2}{2}.$$

$$\frac{\partial f_t(\mathbf{x})}{\partial x_i}=\mathbb{E}[f(R\cup \{i\})-f(R)],\forall i\in [n],$$

$$\sum _{\{z,r\}\in E}\mathbf{x}(r)=\{\begin{array}{ll}a& z=s\\ -a& z=v\\ 0& \text{otherwise}\end{array}$$

$$\sum _{\{z,r\}\in E}\mathbf{x}($$