Algorithms of Inertial Mirror Descent in Convex Problems of Stochastic Optimization

TL;DR

This paper introduces an inertial mirror descent method for convex stochastic optimization problems, extending classical mirror descent with a new approach inspired by the heavy ball method, and provides theoretical error bounds.

Contribution

It proposes a novel inertial mirror descent algorithm that does not require averaging, applicable to convex problems, with proven error bounds and a discrete implementation.

Findings

Inertial MD generalizes classical mirror descent.

The method achieves a proven upper bound on objective function error.

Discrete algorithm implementation is provided.

Abstract

The goal is to modify the known method of mirror descent (MD), proposed by A.S. Nemirovsky and D.B. Yudin in 1979. The paper shows the idea of a new, so-called inertial MD method with the example of a deterministic optimization problem in continuous time. In particular, in the Euclidean case, the heavy ball method by B.T. Polyak is realized. It is noted that the new method does not use additional averaging. A discrete algorithm of inertial MD is described. The theorem on the upper bound on the error in the objective function is proved.

Equations

$$\dot{\zeta }(t)$$

$${\mu }_t\dot{x}(t)+x(t)$$

$$V(x)=\underset{\zeta \in {\mathbb{R}}^{\mathbb{N}}}{\sup }\{⟨\zeta ,x⟩-W(\zeta )\}.$$

$$\dot{x}(t)=-\nabla f(x(t)),t\ge 0.$$

$$\mu \ddot{x}(t)+\dot{x}(t)=-\nabla f(x(t)),t\ge 0.$$

$$W_{\ast }(\zeta )=W(\zeta )-⟨\zeta ,x^{\ast }⟩,\zeta \in {\mathbb{R}}^{\mathbb{N}},$$

$$\frac{d}{dt}{W}_{\ast }(\zeta (t))$$

$$\le$$

$$\int_{0}^{t}f(x(t))dt\,-\,tf^{*}\leq-W_{*}(\zeta(t))-\mu_{t}[f(x(t))-f^{*}]\Big{|}_{0}^{t}+\int_{0}^{t}[f(x(t))-f^{*}]\dot{\mu}_{t}dt,$$

$${\int }_0^{t}f(x(t))dt-t{f}^{\ast }$$

$${\mu }_0=0,{\dot{\mu }}_t\le 1\forall t>0,$$

$$f(x(t))-f^{\ast }\le V(x^{\ast })/{\mu }_t.$$

$${\mu }_t=t,t\ge 0.$$

$$\dot{\zeta }(t)$$

$$t\dot{x}(t)+x(t)$$

$$f(x(t))-f^{\ast }\le V(x^{\ast })t^{-1},\forall t>0.$$

$$f(x)\triangleq \mathbb{E}Q(x,Z)\to \underset{x\in X}{\min },$$

$$u_k(x)={\nabla }_{x}Q(x,Z_k),k=1,2,\dots ,$$

$$\mathbb{E}{u}_k(x)\in \partial f(x).$$

$$\Vert \nabla W_{\beta }(\zeta )-\nabla W_{\beta }(\tilde{\zeta })\Vert \le \frac{1}{\alpha \beta }\Vert \zeta -\tilde{\zeta }{\Vert }_{\ast },\forall \zeta ,\tilde{\zeta }\in E^{\ast },\beta >0,$$

$${\tau }_t$$

$${\zeta }_t$$

$${\tau }_t\frac{x_t-x_{t-1}}{{\gamma }_{t+1}}+x_t$$

$$W_{\beta }(\zeta )=\underset{x\in X}{\sup }\left\{-{\zeta }^{T}x-\beta V(x)\right\},\zeta \in E^{\ast },$$

$$x_t=\frac{{\tau }_t}{{\tau }_t+{\gamma }_{t+1}}{x}_{t-1}-\frac{{\gamma }_{t+1}}{{\tau }_t+{\gamma }_{t+1}}\nabla W_{{\beta }_t}({\zeta }_t).$$

$${\gamma }_i\equiv 1,{\beta }_i={\beta }_0\sqrt{i+1},i=1,2,\dots ,{\beta }_0>0.$$

$${\zeta }_t$$

$$x_t$$

$$\underset{x\in X}{\sup }\mathbb{E}\Vert {\nabla }_{x}Q(x,Z){\Vert }_{\ast }^2\le L_{X,Q}^2,$$

$$\mathbb{E}f(x_t)-\underset{x\in X}{\min }f(x)\le \left({\beta }_{0}V(x^{\ast })+\frac{L_{X,Q}^2}{\alpha {\beta }_0}\right)\frac{\sqrt{t+2}}{t+1}.$$

$$\mathbb{E}f(x_t)-\underset{x\in X}{\min }f(x)\le 2L_{X,Q}{\left({\alpha }^{-1}\overline{V}\right)}^{1/2}\frac{\sqrt{t+2}}{t+1}.$$

Full text

ICS RAS, Moscow, Russia

1. The idea of method of inertial mirror descent

Let $f:\mathbb{R^{N}}\to\mathbb{R}$ be convex, differentiable function having a unique minimum point $x^{*}\in\mathrm{Argmin}f(x)$ and its minimal value $f^{*}=f(x^{*})$. Consider continuous algorithm which extends MDM that is

[TABLE]

Functional parameter in (2) is a convex, continuously differentiable function $W:\mathbb{R^{N}}\to\mathbb{R}_{+}$ having conjugate function

[TABLE]

Let $W(0)=0$, $V(0)=0$, and $\nabla W(0)=0$ for simplicity.

[TABLE]

Under $\mu_{t}\equiv\mu>0$ and $W(\zeta)\equiv\zeta$, algorithm (1)–(2) leads to continuous method of heavy ball (MHB) [9]

[TABLE]

• $\square$*

Further, we assume that differentiable parameter $\mu_{t}\geq 0$, and method (1)–(2) we call Method of Inertial Mirror Descent (MIDM).

Assume a solution $\{x(t)$, $t\geq 0\}$ to system equations (1)–(2) exists.

Consider function

[TABLE]

attempting to find a candidate Lyapunov function.

Trajectory derivative to system (1)–(2) be

[TABLE]

where last inequality results from convexity $f(\cdot)$. Now, integrating on interval $[0,t]$ with $W_{*}(0)=0$, we obtain

[TABLE]

where two last terms in RHS got by integrating in parts. Taking (3) into account, we continue (7):

[TABLE]

Therefore, it is reasonable to introduce the following constraints on patameter $\mu_{t}\geq 0$ :

[TABLE]

leading to inequality

[TABLE]

Maximizing $\mu_{t}$ under constraints (9) we get

[TABLE]

The related (continuous) IMD algorithm

[TABLE]

proves upper bound

[TABLE]

2. Stochastic optimization problem

Consider minimization problem

[TABLE]

where loss function $Q:X\times\mathcal{Z}\to\mathbb{R}_{+}$  contains random variable $Z$ with unknown distribution on space $\mathcal{Z}$, $\mathbb{E}$ — mathematical expectation, set $X\subset{\mathbb{R}}^{N}$ — given convex compact in $N$-dimension space, random function  $Q(\cdot\,,Z):X\to\mathbb{R}_{+}$  is convex a.s. on $X$.

Let i.i.d sample $(Z_{1},\dots,Z_{t-1})$ be given where all $Z_{i}$ have the same distribution on $\mathcal{Z}$ as $Z$. Introduce notation for stochastic subgradients

[TABLE]

such that $\forall x\in X$,

[TABLE]

The goal is in constructing and proving novel recursive MD algorithms meant for minimization (14) and using stochastic subgradients $u_{t}(x_{t-1})$  (15) at current points $x=x_{t-1}\in X$, $t=1,2,\dots$.

3. Algorithm IMD. Main results.

Let $\|\cdot\|$ be a norm in primal space $E=\mathbb{R}^{N}$, and $\|\cdot\|_{*}$ be the related norm in dual space $E^{*}=\mathbb{R}^{N}$; set $X\subset E$ is convex compact.

Assumption (L). Convex function $V:{X}\to\mathbb{R}_{+}$ is such that its $\beta$-conjugate $W_{\beta}$ is continuously differential on $E^{*}$ with gradient $\nabla W_{\beta}$ satisfying Lipschitz condition

[TABLE]

where $\alpha$ is positive constant being independent of $\beta$.

Consider now the discrete time $t\in Z_{+}$. Write a discrete version of algorithm IMD (11)–(12) using stochastic subgradients (15) instead of the gradients $\nabla f(\cdot)$:

[TABLE]

Here function $W_{\beta}$ is defined by proxy-function $V:X\to\mathbb{R}_{+}$ via Legendre–Fenchel transformation, i.e.

[TABLE]

[TABLE]

Since the vectors  $[-\nabla W_{\beta_{t}}(\zeta_{t})]\in X$ under each $t\geq 0$, equations (16)–(17) show that $x_{t}\in X$ by induction. $\Box$

Further, let sequences $(\gamma_{i})_{i\geq 1}$ and $(\beta_{i})_{i\geq 1}$ are of view

[TABLE]

Then system equations (16)–(18) leads to the IMD algorithm:

[TABLE]

[TABLE]

where constant $L_{{X},\,Q}\in(0,\infty)$. Let $V$ be proxy-function on ${X}$ with parameter $\alpha>0$ from assumption (L), and let exists minimum point ${x}^{*}\in\displaystyle\mathop{\mathrm{Arg}\!\min}_{{x}\in{X}}f({x})$. Then for any $t\geq 1$ estimate ${x}_{t}$ , defined by algorithm (22), (23) with stochastic subgradients (15) and sequence $(\beta_{i})_{i\geq 1}$ from (21) with arbitrary $\beta_{0}>0$, satisfies inequality

[TABLE]

$\Box$**

[TABLE]

In particular, one may get $\overline{V}=\displaystyle\max_{{x}\in{X}}V({x})$. $\Box$