Second order dynamical systems with penalty terms associated to monotone inclusions

TL;DR

This paper studies a second order dynamical system in Hilbert spaces involving penalty terms linked to monotone inclusions, establishing existence, uniqueness, and convergence of solutions under certain conditions.

Contribution

It introduces a novel second order dynamical system with penalty terms for monotone inclusions and proves convergence results including strong convergence under strong monotonicity.

Findings

Existence and uniqueness of strong global solutions.

Ergodic asymptotic convergence to a zero of the operator.

Strong convergence when the operator is strongly monotone.

Abstract

In this paper we investigate in a Hilbert space setting a second order dynamical system of the form $$\ddot{x}(t)+\g(t)\dot{x}(t)+x(t)-J_{\lambda(t) A}\big(x(t)-\lambda(t) D(x(t))-\lambda(t)\beta(t)B(x(t))\big)=0,$$ where $A:{\mathcal H}\toto{\mathcal H}$ is a maximal monotone operator, $J_{\lambda(t) A}:{\mathcal H}\To{\mathcal H}$ is the resolvent operator of $\lambda(t)A$ and $D,B: {\mathcal H}\rightarrow{\mathcal H}$ are cocoercive operators, and $\lambda,\beta :[0,+\infty)\rightarrow (0,+\infty)$, and $\gamma:[0,+\infty)\rightarrow (0,+\infty)$ are step size, penalization and, respectively, damping functions, all depending on time. We show the existence and uniqueness of strong global solutions in the framework of the Cauchy-Lipschitz-Picard Theorem and prove ergodic asymptotic convergence for the generated trajectories to a zero of the operator $A+D+{N}_C,$ where $C=\zer B$ and $N_C$ denotes the normal cone operator of $C$. To this end we use Lyapunov analysis combined with the celebrated Opial Lemma in its ergodic continuous version. Furthermore, we show strong convergence for trajectories to the unique zero of $A+D+{N}_C$, provided that $A$ is a strongly monotone operator.

Equations

$$\ddot{x}(t)+\gamma(t)\dot{x}(t)+x(t)-J_{\lambda(t)A}\big{(}x(t)-\lambda(t)D(x(t))-\lambda(t)\beta(t)B(x(t))\big{)}=0,$$

$$\underset{x\in \mathrm{argmin}\psi }{\inf }\{f(x)+g(x)\},$$

$$\underset{x\in \mathcal{H}}{\inf }\{f(x)+g(x)+{\delta }_{\mathrm{argmin}\psi }(x)\}.$$

$$0\in \partial f(x)+\nabla g(x)+\partial {\delta }_{\mathrm{argmin}\psi }(x),$$

$$0\in \partial f(x)+\nabla g(x)+N_{\mathrm{zer}\nabla \psi }(x).$$

$$0\in Ax+Dx+N_C(x),$$

$$\left\{\begin{array}[]{ll}\ddot{x}(t)+\gamma(t)\dot{x}(t)+x(t)=J_{\lambda(t)A}\big{(}x(t)-\lambda(t)D(x(t))-\lambda(t)\beta(t)B(x(t))\big{)}\\ x(0)=u_{0},\,\dot{x}(0)=v_{0},\end{array}\right.$$

$$0\in Ax+Dx.$$

$$\left\{\begin{array}[]{ll}\ddot{x}(t)+\gamma(t)\dot{x}(t)+x(t)=\operatorname*{prox}_{\lambda(t)f}\big{(}x(t)-\lambda(t)\nabla g(x(t))-\lambda(t)\beta(t)\nabla\psi(x(t))\big{)}\\ x(0)=u_{0},\,\dot{x}(0)=v_{0},\end{array}\right.$$

$$z\in \mathrm{zer}A\text{mbox}ifandonlyif⟨u-z,w⟩\ge 0\text{mbox}forall(u,w)\in \mathrm{Gr}A.$$

$${\phi }_A:\mathcal{H}\times \mathcal{H}\to \overline{\mathbb{R}},{\phi }_A(x,u)=\underset{(y,v)\in \mathrm{Gr}A}{\sup }\{⟨x,v⟩+⟨y,u⟩-⟨y,v⟩\},$$

$${\phi }_A(x,u)\ge ⟨x,u⟩\forall (x,u)\in \mathcal{H}\times \mathcal{H},$$

$$\partial f(x)=\{u\in \mathcal{H}:f(y)\ge f(x)+⟨u,y-x⟩\forall y\in \mathcal{H}\}.$$

$$\underset{\gamma f}{\mathrm{prox}}(x)=\underset{y\in \mathcal{H}}{\mathrm{argmin}}\left\{f(y)+\frac{1}{2\gamma }\Vert y-x{\Vert }^2\right\},$$

$$(H1):\gamma ,\lambda ,\beta :[0,+\infty )⟶(0,+\infty )\text{mbox}arecontinuousoneachinterval[0,b],\text{mbox}for0<b<+\infty ,$$

$$\left\{\begin{array}[]{ll}\dot{X}(t)=F(t,X(t))\\ X(0)=(u_{0},v_{0}),\end{array}\right.$$

$$\Vert F(t,u,v)-F(t,\bar{u},\bar{v})\Vert =$$

$$\le$$

$$\Vert J_{\lambda (t)A}(s)-J_{\lambda (t)A}(\bar{s})\Vert \le$$

$$\le$$

$$\Vert F(t,u,v)-F(t,\bar{u},\bar{v})\Vert \le$$

$$\le$$

$$L(t):=\sqrt{5}+2\gamma (t)+\sqrt{2}(1+\lambda (t)L_D+\lambda (t)\beta (t)L_B)$$

$${\int }_0^b\Vert F(t,u,v)\Vert dt=$$

$$\le$$

$$\Vert J_{\lambda (t)A}(u-\lambda (t)D(u)-\lambda (t)\beta (t)B(u))\Vert$$

$$\Vert J_{\lambda (t)A}(u-\lambda (t)D(u)-\lambda (t)\beta (t)B(u))-J_{\lambda (t)A}(u-\underline{\lambda }D(u)-\underline{\lambda }\underline{\beta }B(u))+J_{\lambda (t)A}(u-\underline{\lambda }D(u)-\underline{\lambda }\underline{\beta }B(u))\Vert$$

$$(\lambda (t)-\underline{\lambda })\Vert D(u)\Vert +(\lambda (t)\beta (t)-\underline{\lambda }\underline{\beta })\Vert B(u)\Vert +\Vert J_{\lambda (t)A}(u-\underline{\lambda }D(u)-\underline{\lambda }\underline{\beta }B(u))\Vert .$$

$$\Vert J_{\lambda (t)A}(u-\underline{\lambda }D(u)-\underline{\lambda }\underline{\beta }B(u))\Vert$$

$$\Vert J_{\lambda (t)A}(u-\underline{\lambda }D(u)-\underline{\lambda }\underline{\beta }B(u))-J_{\underline{\lambda }A}(u-\underline{\lambda }D(u)-\underline{\lambda }\underline{\beta }B(u))+J_{\underline{\lambda }A}(u-\underline{\lambda }D(u)-\underline{\lambda }\underline{\beta }B(u))\Vert$$

$$(\lambda (t)-\underline{\lambda })\Vert A_{\underline{\lambda }}(u-\underline{\lambda }D(u)-\underline{\lambda }\underline{\beta }B(u))\Vert +\Vert J_{\underline{\lambda }A}(u-\underline{\lambda }D(u)-\underline{\lambda }\underline{\beta }B(u))\Vert ,$$

$${\int }_0^b\Vert F(t,u,v)\Vert dt\le$$

$$+{\int }_0^b\left(2(\lambda (t)-\underline{\lambda })\Vert A_{\underline{\lambda }}(u-\underline{\lambda }D(u)-\underline{\lambda }\underline{\beta }B(u))\Vert +2\Vert J_{\underline{\lambda }A}(u-\underline{\lambda }D(u)-\underline{\lambda }\underline{\beta }B(u))\Vert \right)dt.$$

$$\frac{d}{dt}F(t)\le G(t).$$

Full text

Second order dynamical systems with penalty terms associated to monotone inclusions

Radu Ioan Boţ University of Vienna, Faculty of Mathematics, Oskar-Morgenstern-Platz 1, A-1090 Vienna, Austria, email: radu.bot@univie.ac.at. Research partially supported by FWF (Austrian Science Fund), project I 2419-N32.   

Ernö Robert Csetnek University of Vienna, Faculty of Mathematics, Oskar-Morgenstern-Platz 1, A-1090 Vienna, Austria, email: ernoe.robert.csetnek@univie.ac.at. Research supported by FWF (Austrian Science Fund), Lise Meitner Programme, project M 1682-N25.   

Szilárd Csaba László Technical University of Cluj-Napoca, Department of Mathematics, Str. Memorandumului nr. 28, 400114 Cluj-Napoca, Romania, e-mail: laszlosziszi@yahoo.com

Abstract. In this paper we investigate in a Hilbert space setting a second order dynamical system of the form

[TABLE]

where $A:{\mathcal{H}}\rightrightarrows{\mathcal{H}}$ is a maximal monotone operator, $J_{\lambda(t)A}:{\mathcal{H}}\longrightarrow{\mathcal{H}}$ is the resolvent operator of $\lambda(t)A$ and $D,B:{\mathcal{H}}\rightarrow{\mathcal{H}}$ are cocoercive operators, and $\lambda,\beta:[0,+\infty)\rightarrow(0,+\infty)$, and $\gamma:[0,+\infty)\rightarrow(0,+\infty)$ are step size, penalization and, respectively, damping functions, all depending on time. We show the existence and uniqueness of strong global solutions in the framework of the Cauchy-Lipschitz-Picard Theorem and prove ergodic asymptotic convergence for the generated trajectories to a zero of the operator $A+D+{N}_{C},$ where $C=\operatorname*{zer}B$ and $N_{C}$ denotes the normal cone operator of $C$. To this end we use Lyapunov analysis combined with the celebrated Opial Lemma in its ergodic continuous version. Furthermore, we show strong convergence for trajectories to the unique zero of $A+D+{N}_{C}$, provided that $A$ is a strongly monotone operator.

Key Words. dynamical systems, Lyapunov analysis, monotone inclusions, penalty schemes

AMS subject classification. 34G25, 47J25, 47H05, 90C25

1 Introduction and preliminaries

Consider the bilevel optimization problem

[TABLE]

where $f:{\mathcal{H}}\longrightarrow{\mathbb{R}}\cup\{+\infty\}$ is a proper convex and lower semicontinuous function, $g,\psi:{\mathcal{H}}\longrightarrow{\mathbb{R}}$ are convex and (Fréchet) differentiable functions both with Lipschitz continuous gradients, and $\operatorname*{argmin}\psi$ denotes the set of global minimizers of $\psi$, assumed to be nonempty.

By making use of the indicator function of $\operatorname*{argmin}\psi$, $\eqref{e1}$ can be rewritten as

[TABLE]

Obviously, $x\in\operatorname*{argmin}\psi$ is an optimal solution of $(\ref{e2})$ if and only if $0\in\partial(f+g+\delta_{\operatorname*{argmin}\psi})(x)$, which can be split in

[TABLE]

provided a suitable qualification condition which guarantees the subdifferential sum rule holds.

Using that $\partial\delta_{\operatorname*{argmin}\psi}(x)={N}_{\operatorname*{argmin}\psi}(x)$ and $\operatorname*{argmin}\psi=\operatorname*{zer}\nabla\psi$, $\eqref{e3}$ is nothing else than

[TABLE]

This motivates us to investigate the following inclusion problem

[TABLE]

where $A:{\mathcal{H}}\rightrightarrows{\mathcal{H}}$ is a maximally monotone operator, $D:{\mathcal{H}}\longrightarrow{\mathcal{H}}$ is a $L_{D}^{-1}$-cocoercive operator, and $B:{\mathcal{H}}\longrightarrow{\mathcal{H}}$ is a $L_{B}^{-1}$-cocoercive operator with $L_{D},L_{B}>0$, $C=\operatorname*{zer}B$, and $N_{C}$ denotes the normal cone operator of the set $C$. We recall that by the classical Baillon-Haddad Theorem, the gradient of a convex and (Fréchet) differentiable function is $L$-Lipschitz continuous, for $L>0$, if and only if it is $L^{-1}$-cocoercive, see for instance [11, Corollary 18.16]).

In [16], a first order dynamical system has been assigned to the monotone inclusion $(\ref{e4})$, and it has been shown that the generated trajectories converge to a solution of it. In this paper, we assign to $(\ref{e4})$ the following second order dynamical system

[TABLE]

where $u_{0},v_{0}\in{\mathcal{H}}$ and $\gamma,\lambda,\beta:[0,+\infty)\longrightarrow(0,+\infty).$ Dynamical systems governed by resolvents of maximally monotone operators have been considered in [1, 2], and then further developed in [17, 19].

The study of second order dynamical systems is motivated by the fact that the presence of the acceleration term $\ddot{x}(t)$ can lead to better convergence properties of the trajectories. Time discretizations of second order dynamical systems give usually rise to numerical algorithms with inertial terms which have been shown to have improved convergence properties (see [23]). The geometric damping function $\gamma$ which acts on the velocity can in some situations accelerate the asymptotic properties of the orbits, as emphasized for example in [28].

For $B=0$ and $\lambda$ is constant, the differential equation (6) becomes the second order forward-backward dynamical system investigated in [19] in relation to the monotone inclusion problem

[TABLE]

On the other hand, when particularized to the monotone inclusion system (4), the differential equation (6) reads

[TABLE]

where we made used of the fact that the resolvent of the subdifferential of a proper, convex and lower semicontinuous function is the proximal point operator of the latter. In case $f=0$, $\gamma$ is constant and $\lambda$ is also constant and identical to $1$, (7) leads to the differential equation that has been investigated in [7] and [18].

The first part of the paper is devoted to the proof of the existence and uniqueness of (locally) absolutely continuous trajectories generated by the dynamical system (6); an important ingredient for this analysis is the Cauchy-Lipschitz-Picard Theorem (see [22, 27]). The proof of the convergence of the trajectories to a solution of (5) is the main result of the manuscript. Provided that a condition expressed in terms of the Fitzpatrick function of the cocoercive operator $B$ is fulfilled, we prove weak ergodic convergence of the orbits. Furthermore, we show that, if the operator $A$ is strongly monotone, then one obtains even strong (non-ergodic) convergence for the generated trajectories.

In the remaining of this section, we explain the notations we used up to this point and will use throughout the paper (see [14, 11, 26]).

The real Hilbert space ${\mathcal{H}}$ is endowed with inner product $\langle\cdot,\cdot\rangle$ and associated norm $\|\cdot\|=\sqrt{\langle\cdot,\cdot\rangle}$. The normal cone of of a set $S\subseteq{\mathcal{H}}$ is defined by $N_{S}(x)=\{u\in{\mathcal{H}}:\langle y-x,u\rangle\leq 0\ \forall y\in S\}$, if $x\in S$ and $N_{S}(x)=\emptyset$ for $x\notin S$. The following characterization of the elements of the normal cone of a nonempty set by means of its support function will be used several times in the paper: for $x\in S$, $u\in N_{S}(x)$ if and only if $\sigma_{S}(u)=\langle x,u\rangle$, where $\sigma_{S}:{\mathcal{H}}\rightarrow{\mathbb{R}}\cup\{+\infty\}$ is defined by $\sigma_{S}(u)=\sup_{y\in S}\langle y,u\rangle$.

Let $A:{\mathcal{H}}\rightrightarrows{\mathcal{H}}$ be a set-valued operator. We denote by $\operatorname*{Gr}A=\{(x,u)\in{\mathcal{H}}\times{\mathcal{H}}:u\in Ax\}$ its graph, by $\operatorname*{dom}A=\{x\in{\mathcal{H}}:Ax\neq\emptyset\}$ its domain and by $\operatorname*{ran}A=\{u\in{\mathcal{H}}:\exists x\in{\mathcal{H}}\mbox{ s.t. }u\in Ax\}$ its range. The notation $\operatorname*{zer}A=\{x\in{\mathcal{H}}:0\in Ax\}$ stands for the set of zeros of the operator $A$. We say that $A$ is monotone if $\langle x-y,u-v\rangle\geq 0$ for all $(x,u),(y,v)\in\operatorname*{Gr}A$. Further, a monotone operator $A$ is said to be maximally monotone, if there exists no proper monotone extension of the graph of $A$ on ${\mathcal{H}}\times{\mathcal{H}}$. The following characterization of the zeros of a maximally monotone operator will be crucial in the asymptotic analysis of (6): if $A$ is maximally monotone, then

[TABLE]

The resolvent of $A$, $J_{A}:{\mathcal{H}}\rightrightarrows{\mathcal{H}}$, is defined by $p\in J_{A}(x)$ if and only if $x\in p+Ap$. Moreover, if $A$ is maximally monotone, then $J_{A}:{\mathcal{H}}\rightarrow{\mathcal{H}}$ is single-valued and maximally monotone (cf. [11, Proposition 23.7 and Corollary 23.10]). We will also use the Yosida approximation of the operator $A$, which is defined for $\alpha>0$ by $A_{\alpha}=\frac{1}{\alpha}(\operatorname*{Id}-J_{\alpha A})$, where $\operatorname*{Id}:{\mathcal{H}}\rightarrow{\mathcal{H}},\operatorname*{Id}(x)=x$ for all $x\in{\mathcal{H}}$, is the identity operator on ${\mathcal{H}}$.

The notion of Fitzpatrick function associated to a monotone operator $A$ will be important in the formulation of the condition under which the convergence of the trajectories is achieved. It is defined as

[TABLE]

and it is a convex and lower semicontinuous function. Introduced by Fitzpatrick in [21], this notion played in the last years a crucial role in the investigation of maximality of monotone operators by means of convex analysis specific tools (see [11, 12, 14, 26] and the references therein). We notice that, if $A$ is maximally monotone, then $\varphi_{A}$ is proper and

[TABLE]

with equality if and only if $(x,u)\in\operatorname*{Gr}A$. We refer the reader to [12] for explicit formulae of Fitzpatrick functions associated to particular classes of monotone operators.

Let $\gamma>0$ be arbitrary. A single-valued operator $A:{\mathcal{H}}\rightarrow{\mathcal{H}}$ is said to be $\gamma$-cocoercive, if $\langle x-y,Ax-Ay\rangle\geq\gamma\|Ax-Ay\|^{2}$ for all $(x,y)\in{\mathcal{H}}\times{\mathcal{H}}$, and $\gamma$-Lipschitz continuous, if $\|Ax-Ay\|\leq\gamma\|x-y\|$ for all $(x,y)\in{\mathcal{H}}\times{\mathcal{H}}$.

For a proper, convex and lower semicontinuous function $f:{\mathcal{H}}\rightarrow{\mathbb{R}}\cup\{+\infty\}$, its (convex) subdifferential at $x\in{\mathcal{H}}$ is defined as

[TABLE]

When seen as a set-valued mapping, it is a maximally monotone operator and its resolvent is given by $J_{\gamma\partial f}=\operatorname*{prox}_{\gamma f}$ (see [11]), where $\operatorname*{prox}_{\gamma f}:{\mathcal{H}}\rightarrow{\mathcal{H}}$,

[TABLE]

denotes the proximal point operator of $f$.

2 Existence and uniqueness of the trajectory

We start by specifying which type of solutions are we considering in the analysis of the dynamical system (6).

Definition 1**.**

We say that $x:[0,+\infty)\rightarrow{{\mathcal{H}}}$ is a strong global solution of (6), if the following properties are satisfied:

(i) $x,\dot{x}:[0,+\infty)\rightarrow{{\mathcal{H}}}$ are locally absolutely continuous, in other words, absolutely continuous on each interval $[0,b]$ for $0<b<+\infty$;

(ii) \ddot{x}(t)+\gamma(t)\dot{x}(t)+\big{(}x(t)-J_{\lambda(t)A}(x(t)-\lambda(t)D(x(t))-\lambda(t)\beta(t)B(x(t)))\big{)}=0 for almost every $t\geq 0$;

(iii) $x(0)=u_{0}$ and $\dot{x}(0)=v_{0}$.

For proving existence and uniqueness of the strong global solutions of (6), we use the Cauchy-Lipschitz-Picard Theorem for absolutely continues trajectories (see for example [22, Proposition 6.2.1], [27, Theorem 54]). The key argument is that one can rewrite (6) as a particular first order dynamical system in a suitably chosen product space (see also [3]).

To this end we make the following assumption:

[TABLE]

which also describes the framework in which we will carry out the convergence analysis in the forthcoming sections.

Theorem 2**.**

Suppose that $\gamma,\lambda$ and $\beta$ satisfy $(H1)$. Then for every $u_{0},v_{0}\in{\mathcal{H}}$ there exists a unique strong global solution of $(\ref{e5}).$

Proof.

Define $X:[0,+\infty)\longrightarrow{\mathcal{H}}\times{\mathcal{H}}$ as $X(t)=(x(t),\dot{x}(t)).$ Then (6) is equivalent to

[TABLE]

where F(t,u,v)=\big{(}v,-\gamma(t)v-u+J_{\lambda(t)A}(u-\lambda(t)D(u)-\lambda(t)\beta(t)B(u))\big{)}.

First we show that $F(t,\cdot,\cdot)$ is Lipschitz continuous with a Lipschitz constant $L(t)\in L^{1}_{loc}([0,+\infty))$, for every $t\geq 0$. Indeed,

[TABLE]

where $s=u-\lambda(t)D(u)-\lambda(t)\beta(t)B(u)$ and $\bar{s}=\bar{u}-\lambda(t)D(\bar{u})-\lambda(t)\beta(t)B(\bar{u}).$

By using the nonexpansivity of $J_{\lambda(t)A}$ we get

[TABLE]

Hence,

[TABLE]

Since $\gamma,\lambda,\beta\in L^{1}_{loc}([0,+\infty))$, it follows that

[TABLE]

is also locally integrable on $[0,+\infty)$.

Next we show that $F(\cdot,u,v)\in L^{1}_{loc}([0,+\infty),{\mathcal{H}}\times{\mathcal{H}})$ for all $u,v\in{\mathcal{H}}.$ We fix $u,v\in{\mathcal{H}}$ and $b>0$, and notice that

[TABLE]

According to $(H1)$, there exist positive numbers $\underline{\lambda}$ and $\underline{\beta}$ such that $0<\underline{\lambda}\leq\lambda(t)$ and $0<\underline{\beta}\leq\beta(t)$ for all $t\in[0,b]$. Hence

[TABLE]

In addition,

[TABLE]

where the last inequality follows from the Lipschitz property of the resolvent operator as a function of the step size, which basically follows by combining [20, Proposition 2.6] and [11, Proposition 23.28] (see also [2, Proposition 3.1]). Hence,

[TABLE]

Hence, $F(\cdot,u,v)\in L^{1}_{loc}([0,+\infty),{\mathcal{H}}\times{\mathcal{H}})$ for all $u,v\in{\mathcal{H}}.$ The conclusion of the theorem follows by applying the Cauchy-Lipschitz-Picard theorem to the first order dynamical system (9).

∎

3 Some preparatory lemmas

In this section we provide some preparatory lemmas which will be used when proving the convergence of the trajectories generated by the dynamical system (6). We start by recalling two central results; see for example [2, Lemma 5.1] and [2, Lemma 5.2], respectively.

Lemma 3**.**

Suppose that $F:[0,+\infty)\rightarrow{\mathbb{R}}$ is locally absolutely continuous and bounded below and that there exists $G\in L^{1}([0,+\infty))$ such that for almost every $t\in[0,+\infty)$

[TABLE]

Then there exists $\lim_{t\rightarrow\infty}F(t)\in{\mathbb{R}}$.

Lemma 4**.**

If $1\leq p<\infty$, $1\leq r\leq\infty$, $F:[0,+\infty)\rightarrow[0,+\infty)$ is locally absolutely continuous, $F\in L^{p}([0,+\infty))$, $G:[0,+\infty)\rightarrow{\mathbb{R}}$, $G\in L^{r}([0,+\infty))$ and for almost every $t\in[0,+\infty)$

[TABLE]

then $\lim_{t\rightarrow+\infty}F(t)=0$.

Lemma 5**.**

Suppose that $(H1)$ holds and let $x$ be the unique strong global solution of (6). Take $(x^{*},w)\in\operatorname*{Gr}(A+D+N_{C})$ such that $w=v+Dx^{*}+p$, where $v\in Ax^{*}$ and $p\in N_{C}(x^{*}).$ For every $t\geq 0$ consider the function $h(t)=\frac{1}{2}\|x(t)-x^{*}\|^{2}.$ Then the following inequality holds for almost every $t\geq 0$:

[TABLE]

Proof.

We have $\dot{h}(t)=\langle\dot{x}(t),x(t)-x^{*}\rangle$ and $\ddot{h}(t)=\langle\ddot{x}(t),x(t)-x^{*}\rangle+\|\dot{x}(t)\|^{2}$ for every $t\geq 0$. By using the definition of the resolvent, the differential equation in (6) can be written for almost every $t\geq 0$ as

[TABLE]

or, equivalently,

[TABLE]

Since $v\in Ax^{*}$ and $A$ is monotone, we get for almost every $t\geq 0$

[TABLE]

It follows that

[TABLE]

for almost every $t\geq 0$. Hence, for almost every $t\geq 0$

[TABLE]

and from here, by using mean inequalities,

[TABLE]

Since $v=w-Dx^{*}-p$, we obtain for the first summand of the term on the right-hand side of the above inequality for every $t\geq 0$ the following estimate

[TABLE]

Hence, for almost every $t\geq 0$, we have

[TABLE]

which is nothing else than the desired conclusion. ∎

Lemma 6**.**

Suppose that $(H1)$ holds and let $x$ be the unique strong global solution of (6). Take $x^{*}\in C\cap\operatorname*{dom}A$ and $v\in Ax^{*}.$ For every $t\geq 0$ consider the function $h(t)=\frac{1}{2}\|x(t)-x^{*}\|^{2}.$ Then for every $\epsilon>0$ the following inequality holds for almost every $t\geq 0$:

[TABLE]

Proof.

Let be $\epsilon>0$ fixed. According to (3) in the proof of the above lemma, we have for almost every $t\geq 0$

[TABLE]

Since $B$ is $\frac{1}{L_{B}}$-cocoercive and $Bx^{*}=0$ we have $\langle B(x(t)),x^{*}-x(t)\rangle\leq-\frac{1}{L_{B}}\|B(x(t))\|^{2}$, hence

[TABLE]

for every $t\geq 0$. Consequently,

[TABLE]

which, combined with

[TABLE]

implies for almost every $t\geq 0$ relation (6). ∎

Lemma 7**.**

Suppose that $(H1)$ holds and let $x$ be the unique strong global solution of (6). Furthermore, suppose that $\lim\sup_{t\longrightarrow+\infty}\lambda(t)\beta(t)<\frac{1}{L_{B}}.$ Take $x^{*}\in C\cap\operatorname*{dom}A$ and $v\in Ax^{*}.$ For every $t\geq 0$ consider the function $h(t)=\frac{1}{2}\|x(t)-x^{*}\|^{2}.$ Then there exist $a,b,c>0$ and $t_{0}>0$ such that for almost every $t\geq t_{0}$ the following inequality holds:

[TABLE]

Proof.

Let be $\epsilon>0$. According to the previous lemma, (6) holds for almost every $t\geq 0$. We estimate the last summand in the right-hand side of (6) by using the mean inequality and the cocoercieveness of $D$. For every $t\geq 0$ we obtain

[TABLE]

which, combined with $\|D(x(t))+v\|^{2}\leq 2\|D(x(t))-Dx^{*}\|^{2}+2\|Dx^{*}+v\|^{2}$, implies

[TABLE]

Using the above estimate in (6), we obtain for almost every $t\geq 0$

[TABLE]

or, equivalently

[TABLE]

Since $\lim\sup_{t\longrightarrow+\infty}\lambda(t)\beta(t)<\frac{1}{L_{B}}$, there exists $t_{0}>0$ such that

[TABLE]

for every $t\geq t_{0}.$ Further, we notice that

[TABLE]

for every $\epsilon\in\left(0,\sqrt{(1+L_{B})^{2}+1}-(1+L_{B})\right].$ By chosing $\epsilon_{0}$ from this interval and defining

[TABLE]

the conclusion follows. ∎

Remark 8**.**

In the proof of the above theorem, the choice $\epsilon_{0}\leq\sqrt{(1+L_{B})^{2}+1}-(1+L_{B})<\sqrt{2}-1$, implies that $a<1-\frac{1}{\sqrt{2}}$ and $\frac{1}{2}<c<\frac{3}{4}-\frac{\sqrt{2}}{8}.$**

Lemma 9**.**

Suppose that $(H1)$ holds and let $x$ be the unique strong global solution of (6). Furthermore, suppose that $\lim\sup_{t\longrightarrow+\infty}\lambda(t)\beta(t)<\frac{1}{L_{B}}$ and $\lim_{t\longrightarrow+\infty}\lambda(t)=0$. Take $(x^{*},w)\in\operatorname*{Gr}(A+D+N_{C})$ such that $w=v+Dx^{*}+p$, where $v\in Ax^{*}$ and $p\in N_{C}(x^{*}).$ For every $t\geq 0$ consider the function $h(t)=\frac{1}{2}\|x(t)-x^{*}\|^{2}.$ Then there exist $a,b,c>0$ and $t_{1}>0$ such that for almost every $t\geq t_{1}$ the following inequality holds:

[TABLE]

Proof.

According to Lemma 7, there exist $a,b,c>0$ and $t_{0}>0$ such that for almost every $t\geq t_{0}$ the inequality (7) holds. Since $\lim_{t\longrightarrow+\infty}\lambda(t)=0$, there exists $t_{1}\geq t_{0}$ such that $\lambda(t)\leq\frac{1}{bL_{D}}$, hence $b\lambda^{2}(t)-\frac{\lambda(t)}{L_{D}}\leq 0$ for every $t\geq t_{1}.$ Consequently, we can omit for every $t\geq t_{1}$ the term $\left(b\lambda^{2}(t)-\frac{\lambda(t)}{L_{D}}\right)\|D(x(t))-Dx^{*}\|^{2}$ in (7) and obtain that the inequality

[TABLE]

holds for almost every $t\geq t_{1}$.

Since $Dx^{*}+v=w-p$, we have for every $t\geq 0$

[TABLE]

On the other hand, (3) can be equivalently written for almost every $t\geq t_{1}$ as

[TABLE]

hence

[TABLE]

∎

4 Main result: the convergence of the trajectories

For the proof of the convergence of the trajectories generated by the dynamical system (6) we will utilize the following assumptions:

$(H2):A+N_{C}\mbox{ is maximally monotone and }\operatorname*{zer}(A+D+N_{C})\neq\emptyset$;

$(H3):\lambda\in L^{2}([0,+\infty))\setminus L^{1}([0,+\infty))\mbox{ and }\lim_{t\longrightarrow+\infty}\lambda(t)=0$;

$(H_{fitz}):\ \mbox{For every }p\in\operatorname*{ran}N_{C},\,\int_{0}^{+\infty}\lambda(t)\beta(t)\left(\sup_{u\in C}\varphi_{B}\left(u,\frac{p}{\beta(t)}\right)-\sigma_{C}\left(\frac{p}{\beta(t)}\right)\right)dt<+\infty.$

Remark 10**.**

(i) The first assumption in $(H2)$ is fulfilled when a regularity condition which ensures the maximality of the sum of two maximally monotone operators holds. This is a widely studied topic in the literature; we refer the reader to [11, 13, 14, 26] for such conditions, including the classical Rockafellar’s condition expressed in terms of the domains of the involved operators.

(ii) With respect to $(H_{fitz})$, we would like to remind that a similar condition formulated in terms of the Fitzpatrick function has been considered for the first time in [15] in the discrete setting. Its continuous version has been introduced in [16] and further used also in [5].

This class of conditions, widely used in the context of penalization approaches, has its origin in [6]. Here, in the particular case $C=\operatorname*{argmin}\psi$, where $\psi:{\cal H}\rightarrow{\mathbb{R}}$ is a convex and differentiable function with Lipschitz continuous gradient and such that $\min\psi=0$, the condition

$(H)$: For every $p\in\operatorname*{ran}N_{C}$, $\int_{0}^{+\infty}\lambda(t)\beta(t)\left[\psi^{*}\left(\frac{p}{\beta(t)}\right)-\sigma_{C}\left(\frac{p}{\beta(t)}\right)\right]dt<+\infty$.

has been used in the asymptotic analysis of a coupled dynamical system with multiscale aspects. The function $\psi^{*}:{\cal H}\rightarrow{\mathbb{R}}\cup\{+\infty\}$, $\psi^{*}(u)=\sup_{x\in{\cal H}}\{\langle u,x\rangle-\psi(x)\}$, denotes the Fenchel conjugate of $\psi$.

According to [12], it holds

[TABLE]

Since $\psi(x)=0$ for $x\in C$, condition $(H_{fitz})$ applied to $B=\nabla\psi$ is fulfilled, provided that $(H)$ is fulfilled. For several particular situations where $(H_{fitz})$ is verified (in its continuous or discrete version) we refer the reader to [6, 9, 8, 10, 25, 24].**

For proving the convergence of the trajectories generated by the dynamical system (6) we will also make use of the following ergodic version of the continuous Opial Lemma (see [6, Lemma 2.3]).

Lemma 11**.**

Let $S\subseteq{{\mathcal{H}}}$ be a nonempty set, $x:[0,+\infty)\rightarrow{{\mathcal{H}}}$ a given map and $\lambda:[0,+\infty)\rightarrow(0,+\infty)$ such that $\int_{0}^{+\infty}\lambda(t)=+\infty$. Define $\tilde{x}:[0,+\infty)\rightarrow{{\mathcal{H}}}$ by

[TABLE]

Assume that

(i) for every $z\in S$, $\lim_{t\rightarrow+\infty}\|x(t)-z\|$ exists;

(ii) every weak sequential cluster point of the map $\tilde{x}$ belongs to $S$.

Then there exists $x_{\infty}\in S$ such that $w-\lim_{t\rightarrow+\infty}\tilde{x}(t)=x_{\infty}$.

We can state now the main result of this paper.

Theorem 12**.**

Suppose that $(H1)-(H3)$ and $(H_{fitz})$ hold, and let $x$ be the unique strong global solution of (6). Furthermore, suppose that $\lim\sup_{t\longrightarrow+\infty}\lambda(t)\beta(t)<\frac{1}{L_{B}}$, $\gamma$ is locally absolutely continuous and for almost every $t\geq 0$ it holds $\gamma(t)\geq\sqrt{2}$ and $\dot{\gamma}(t)\leq 0$. Let $\tilde{x}:[0,+\infty)\longrightarrow{\mathcal{H}}$ be defined by

[TABLE]

Then the following statements hold:

• (i)

for every $x^{*}\in\operatorname*{zer}(A+D+N_{C}),$ $\|x(t)-x^{*}\|$ converges as $t\longrightarrow+\infty$; in addition, $\dot{x},\ddot{x}\in L^{2}([0,+\infty),{\mathcal{H}}),$ $\lambda(\cdot)\beta(\cdot)\|B(x(\cdot))\|^{2}\in L^{1}([0,+\infty))$, $\int_{0}^{+\infty}\lambda(t)\beta(t)\langle B(x(t)),x(t)-x^{*}\rangle dt<+\infty$, and $\lim_{t\to+\infty}\dot{x}(t)=\lim_{t\to+\infty}\dot{h}(t)=0,$ where $h(t)=\frac{1}{2}\|x(t)-x^{*}\|^{2};$

• (ii)

$\tilde{x}(t)$* converges weakly as $t\longrightarrow+\infty$ to an element in $\operatorname*{zer}(A+D+N_{C})$;*

• (iii)

if, additionally, $A$ is strongly monotone, then $x(t)$ converges strongly as $t\longrightarrow+\infty$ to the unique element of $\operatorname*{zer}(A+D+N_{C})$.

Proof.

(i) Let be $x^{*}\in\operatorname*{zer}(A+D+N_{C})$, thus $(x^{*},0)\in\operatorname*{Gr}(A+D+N_{C})$ and $0=v+Dx^{*}+p$ for $v\in Ax^{*}$ and $p\in N_{C}(x^{*})$. According to Lemma 9 and Remark 8, there exist $a,b,c>0$, with $a<1-\frac{1}{\sqrt{2}}$ and $\frac{1}{2}<c<\frac{3}{4}-\frac{\sqrt{2}}{8}$, and $t_{1}>0$ such that for almost every $t\geq t_{1}$ it holds

[TABLE]

On the other hand, since

[TABLE]

it holds

[TABLE]

for every $t\geq 0$.

By combining these two inequalities, we obtain for almost every $t\geq t_{1}$

[TABLE]

Since $\gamma(t)\geq\sqrt{2}$ and $c>\frac{1}{2}$ one has $c\gamma^{2}(t)-c\dot{\gamma}(t)-1\geq 2c-1>0$ for almost every $t\geq 0$. By using that $\langle B(x(t)),x(t)-x^{*}\rangle\geq\frac{1}{L_{B}}\|B(x(t))\|^{2}$ for every $t\geq 0$ and by neglecting the nonnegative terms on the left-hand side of (4), we get for almost every $t\geq t_{1}$

[TABLE]

Further, by integration, we easily derive that there exists $M>0$ such that for every $t\geq 0$

[TABLE]

Hence, $\dot{h}(t)+\gamma(t)h(t)\leq M$, which leads to $\dot{h}(t)+\sqrt{2}h(t)\leq M$ for all $t\geq 0.$ Consequently, $\frac{d}{dt}(h(t)e^{\sqrt{2}t})\leq Me^{\sqrt{2}t}$; therefore, by integrating this inequality from [math] to $T>0$, one obtains

[TABLE]

which shows that $h$ is bounded, hence $x$ is bounded. Combining this with

[TABLE]

which is a consequence of (19), we derive that $\dot{x}$ is bounded, too.

In conclusion, $t\mapsto\dot{h}(t)+\gamma(t)h(t)+c\gamma(t)\|\dot{x}(t)\|^{2}$ is bounded from below. By taking into account relation (4) and applying Lemma 3, we obtain

[TABLE]

and

[TABLE]

Since

[TABLE]

for every $t\geq 0$ and the function on the right-hand side of the above inequality belongs to $L^{1}([0,+\infty))$, according to Lemma 4 one has $\lim_{t\to+\infty}\dot{x}(t)=0.$ Further, the equality $\dot{h}(t)=\langle\dot{x}(t),x(t)-x^{*}\rangle$ leads to $-\|\dot{x}(t)\|\|x(t)-x^{*}\|\leq\dot{h}(t)\leq\|\dot{x}(t)\|\|x(t)-x^{*}\|$ for every $t\geq 0$. Since $\lim_{t\to+\infty}\dot{x}(t)=0$ and $\|x(\cdot)-x^{*}\|$ is bounded, one obtains $\lim_{t\to+\infty}\dot{h}(t)=0.$

From $\lim_{t\longrightarrow+\infty}(\dot{h}(t)+\gamma(t)h(t)+c\gamma(t)\|\dot{x}(t)\|^{2})\in{\mathbb{R}}$, $\lim_{t\longrightarrow+\infty}\dot{h}(t)=0$ and $\lim_{t\longrightarrow+\infty}c\gamma(t)\|\dot{x}(t)\|^{2}=0$, one obtains that the limit $\lim_{t\longrightarrow+\infty}\gamma(t)h(t)$ exists and it is a finite number. On the other hand, since the limit $\lim_{t\longrightarrow+\infty}\gamma(t)\geq\sqrt{2}$ exists and it is a positive number, one can conclude that $\lim_{t\longrightarrow+\infty}h(t)$ exists and it is finite. Consequently, $\|x(t)-x^{*}\|$ converges as $t\longrightarrow+\infty.$

(ii) We show that every weak sequential limit point of $\tilde{x}$ belongs to $\operatorname*{zer}(A+D+N_{C}).$ Indeed, let $x_{0}$ be a weak sequential limit point of $\tilde{x}$; thus, there exists a sequence $(s_{n})_{n\geq 0}$ with $s_{n}\longrightarrow+\infty$ and $\tilde{x}(s_{n})\longrightarrow x_{0}$ as $n\longrightarrow+\infty.$

Take an arbitrary $(x^{*},w)\in\operatorname*{Gr}(A+D+N_{C})$ with $w=v+Dx^{*}+p$, $v\in Ax^{*}$ and $p\in N_{C}(x^{*})$. Since $\lim_{t\longrightarrow+\infty}\lambda(t)=0$, there exists $t_{2}>0$ such that for every $t\geq t_{2}$ one has $\lambda(t)\left(\frac{1}{L_{D}}-\lambda(t)\right)\geq 0$. From (5) we obtain

[TABLE]

for every $t\geq t_{2}$. By integrating from $t_{2}$ to $T>t_{2}$, we get from here

[TABLE]

where

[TABLE]

Since $\gamma(t)\dot{h}(t)\geq\frac{d}{dt}(\gamma(t)h(t))$ and $\gamma(T)h(T)\geq 0$, we obtain

[TABLE]

hence

[TABLE]

where $L_{1}:=L+\dot{h}(t_{2})+\left\langle w,\int_{0}^{t_{2}}\lambda(t)x(t)dt-\left(\int_{0}^{t_{2}}\lambda(t)dt\right)x^{*}\right\rangle\in{\mathbb{R}}.$

We choose in the above inequality $T=s_{n}$ for those $n$ for which $s_{n}>t_{2}$, let $n$ converge to $+\infty$ and so, by taking into account that $\int_{0}^{+\infty}\lambda(t)dt=+\infty$ and $\dot{h}$ is bounded, we obtain

[TABLE]

Since $(x^{*},w)\in\operatorname*{Gr}(A+D+N_{C})$ was arbitrary chosen, it follows that $x_{0}\in\operatorname*{zer}(A+D+N_{C}).$ Hence, by Lemma 11, $\tilde{x}(t)$ converges weakly as $t\longrightarrow+\infty$ to an element in $\operatorname*{zer}(A+D+N_{C})$.

(iii) Assume now that $A$ is strongly monotone, i.e. there exists $\eta>0$ such that

[TABLE]

Let $x^{*}$ be the unique element of $\operatorname*{zer}(A+D+N_{C}).$ Then $0=v+Dx^{*}+p$, for $v\in Ax^{*}$ and $p\in N_{C}(x^{*}).$

Since $v\in Ax^{*}$ and $A$ is $\eta-$strongly monotone, from (11) we obtain for almost every $t\geq 0$

[TABLE]

By repeating the arguments in the proof of Lemma 5, we easily derive for almost every $t\geq 0$

[TABLE]

Since $\lim_{t\longrightarrow+\infty}\lambda(t)=0$, there exists $t_{3}>0$ such that $\lambda^{2}(t)-\frac{\lambda(t)}{L_{D}}\leq 0$ for every $t\geq t_{3}$. Thus for almost every $t\geq t_{3}$

[TABLE]

Combining this inequality with

[TABLE]

we derive for almost every $t\geq t_{3}$

[TABLE]

By using that $\gamma(t)\dot{h}(t)\geq\frac{d}{dt}(\gamma(t)h(t))$ and $\|\ddot{x}(t)+\gamma(t)\dot{x}(t)\|^{2}\leq 2\|\ddot{x}(t)\|^{2}+2\gamma^{2}(t)\|\dot{x}(t)\|^{2}$ for every $t\geq 0$, by integrating from $t_{3}$ to $T>t_{3}$, by letting $T$ converge to $+\infty$, and by using (i), we obtain

[TABLE]

Since $\lambda\in L^{2}([0,+\infty))\setminus L^{1}([0,+\infty))$, it follows that $\|x(t)-x^{*}\|\longrightarrow 0$ as $t\longrightarrow+\infty.$ ∎

Remark 13**.**

We want to emphasize that unlike other papers addressing the asymptotic analysis of second order dynamical systems, and where the damping function $\gamma(t)$ was assumed to be strictly greater than $\sqrt{2}$ for all $t\geq 0$ (see [4, 19]), in Theorem 12 we allow for it take this value, too.**

We close the paper by formulating Theorem 12 in the context of the optimization problem (1), where we also use the relation between the assumptions $(H)$ and $(H_{fitz})$ (see Remark 10).

Corollary 14**.**

Consider the optimization problem (1). Suppose that its system of optimality conditions

[TABLE]

is solvable, that $(H1)-(H3)$ and $(H)$ hold, and let $x$ be the unique strong global solution of (7). Furthermore, suppose that $\lim\sup_{t\longrightarrow+\infty}\lambda(t)\beta(t)<\frac{1}{L_{B}}$, $\gamma$ is locally absolutely continuous and for almost every $t\geq 0$ it holds $\gamma(t)\geq\sqrt{2}$ and $\dot{\gamma}(t)\leq 0$. Let $\tilde{x}:[0,+\infty)\longrightarrow{\mathcal{H}}$ be defined by

[TABLE]

Then the following statements hold:

• (i)

for every $x^{*}\in\operatorname*{zer}(\partial f+\nabla g+N_{\operatorname*{zer}\nabla\psi}),$ $\|x(t)-x^{*}\|$ converges as $t\longrightarrow+\infty$; in addition, $\dot{x},\ddot{x}\in L^{2}([0,+\infty),{\mathcal{H}}),$ $\lambda(\cdot)\beta(\cdot)\|\nabla\psi(x(\cdot))\|^{2}\in L^{1}([0,+\infty))$, $\int_{0}^{+\infty}\lambda(t)\beta(t)\langle\nabla\psi(x(t)),x(t)-x^{*}\rangle dt<+\infty$, and $\lim_{t\to+\infty}\dot{x}(t)=\lim_{t\to+\infty}\dot{h}(t)=0,$ where $h(t)=\frac{1}{2}\|x(t)-x^{*}\|^{2};$

• (ii)

$\tilde{x}(t)$* converges weakly as $t\longrightarrow+\infty$ to an element in $\operatorname*{zer}(\partial f+\nabla g+N_{\operatorname*{zer}\nabla\psi})$, which is also an optimal solution of (1);*

• (iii)

if, additionally, $f$ is strongly convex, then $x(t)$ converges strongly as $t\longrightarrow+\infty$ to the unique element of $\operatorname*{zer}(\partial f+\nabla g+N_{\operatorname*{zer}\nabla\psi})$, which is the unique optimal solution of (1).