On the Relation Between Two Approaches to Necessary Optimality Conditions in Problems with State Constraints

TL;DR

This paper explores two methods for deriving necessary optimality conditions in control problems with state constraints, demonstrating how differentiating along boundary subarcs yields comprehensive stationarity conditions including measure sign definiteness.

Contribution

It introduces a two-stage variation approach to derive full stationarity conditions, linking Gamkrelidze's and Dubovitskii-Milyutin's methods for problems with boundary state constraints.

Findings

Full stationarity conditions include measure sign definiteness.

Differentiating along boundary subarc simplifies the problem.

Two-stage variation approach clarifies the relation between methods.

Abstract

We consider a class of optimal control problems with a state constraint and investigate a trajectory with a single boundary interval (subarc). Following R.V. Gamkrelidze, we differentiate the state constraint along the boundary subarc, thus reducing the original problem to a problem with mixed control-state constraints, and show that this way allows one to obtain the full system of stationarity conditions in the form of A.Ya. Dubovitskii and A.A. Milyutin, including the sign definiteness of the measure (state constraint multiplier), i.e., the nonnegativity of its density and atoms at junction points. The stationarity conditions are obtained by a two-stage variation approach, proposed in this paper. At the first stage, we consider only those variations, which do not affect the boundary interval, and obtain optimality conditions in the form of Gamkrelidze. At the second stage, the variations are concentrated on the boundary interval, thus making possible to specify the stationarity conditions and obtain the sign of density and atoms of the measure.

Equations

$$\text{mbox}ProblemA:\{\begin{array}{l}\begin{array}{rl}\dot{z}=f(z,x,u),& J_A=J(z(0),z(T),x(0),x(T))\to \min ,\\ \dot{x}=g(z,x,u),& {\phi }_s\left(u(t)\right)\le 0,s=1,\dots ,d(\phi ),\\ x(t)\ge 0.& \end{array}\end{array}$$

$${\dot{x}}^0(t_1^0-0)$$

$${\dot{x}}^0(t_2^0+0)$$

$$|z\left(t\right)-z^0\left(\sigma (t)\right)|<\epsilon ,|x\left(t\right)-x^0\left(\sigma (t)\right)|$$

$$|u(t)-u^0\left(\sigma (t)\right)|$$

$$\frac{d{r}_i}{d\tau }={\rho }_i(\tau )f(r_i,y_i,v_i),\frac{d{y}_i}{d\tau }={\rho }_i(\tau )g(r_i,y_i,v_i),i=1,2,3.$$

$$\begin{array}[]{c}r_{1}(1)-r_{2}(0)=0,\qquad y_{1}(1)-y_{2}(0)=0,\qquad t_{1}(1)-t_{2}(0)=0,\\[4.0pt] r_{2}(1)-r_{3}(0)=0,\qquad y_{2}(1)-y_{3}(0)=0,\qquad t_{2}(1)-t_{3}(0)=0.\end{array}$$

$$y_2(0)\ge 0,\frac{d{y}_2}{d\tau }\equiv 0,\text{mbox}i.e.,g(r_2,y_2,v_2)\equiv 0,$$

$$\phi (v_i(\tau ))\le 0,{\rho }_i>0,i=1,2,3.$$

$$J_B:=J\left(r_1(0),r_3(1),y_1(0),y_3(1)\right)\to \min ,$$

$$\{\begin{array}{l}\begin{array}{rlrl}\frac{d{r}_1}{d\tau }& ={\rho }_{1}f(r_1,y_1,v_1),& r_1(1)-r_2(0)& =0,\\ \frac{d{y}_1}{d\tau }& ={\rho }_{1}g(r_1,y_1,v_1),& y_1(1)-y_2(0)& =0,\\ \frac{d{t}_1}{d\tau }& ={\rho }_1,t_1(0)=0,& t_1(1)-t_2(0)& =0,\end{array}\end{array}$$

$$\{\begin{array}{l}\begin{array}{rlrlr}\frac{d{r}_2}{d\tau }& ={\rho }_{2}f(r_2,y_2,v_2),& r_2(1)-r_3(0)& =0,& \\ \frac{d{y}_2}{d\tau }& ={\rho }_{2}g(r_2,y_2,v_2),& y_2(1)-y_3(0)& =0,& y_2(0)\ge 0,\\ \frac{d{t}_2}{d\tau }& ={\rho }_2,& t_2(1)-t_3(0)& =0,& \end{array}\end{array}$$

$$\{\begin{array}{l}\begin{array}{rlr}\frac{d{r}_3}{d\tau }& ={\rho }_{3}f(r_3,y_3,v_3),& \\ \frac{d{y}_3}{d\tau }& ={\rho }_{3}g(r_2,y_2,v_2),& \\ \frac{d{t}_3}{d\tau }& ={\rho }_3,& t_3(1)-T=0,\end{array}\end{array}$$

$$g(r_2,y_2,v_2)\equiv 0,\phi (v_1(\tau ))\le 0,\phi (v_3(\tau ))\le 0.$$

$$g\left(r_1^0(\tau ),y_1^0(\tau ),v_1^0(\tau )\right)\le -c<0\text{mbox}on[\theta ,1].$$

$$\frac{d{y}_1}{d\tau }={\rho }_1(\tau )g\left(r_1(\tau ),y_1(\tau ),v_1(\tau )\right)\le -{\rho }_1(\tau )\frac{c}{2}<0\text{mbox}on[\theta ,1].$$

$$x(t)>0\text{mbox}on[0,t_1)\cup (t_2,T]\text{mbox}andx(t)\ge 0\text{mbox}on[t_1,t_2],$$

$$\sum {\alpha }_i{\Phi }_{iu}^{\prime }(t,x,u)+\sum {\beta }_j{G}_{ju}^{\prime }(t,x,u)=0.$$

$$|{\alpha }_0|+|{\alpha }_1|+\sum |{\beta }_j|+{\int }_0^1|h_1(\tau )|d\tau +{\int }_0^1|\sigma (\tau )|d\tau +{\int }_0^1|h_3(\tau )|d\tau >0,$$

$${\alpha }_0\ge 0,{\alpha }_1\ge 0,h_1(\tau )\ge 0,h_3(\tau )\ge 0,$$

$${\alpha }_1{y}_2(0)=0,h_1(\tau )\phi (v_1^0(\tau ))=0,h_3(\tau )\phi (v_3^0(\tau ))=0,$$

$$\begin{gathered} l={\alpha }_{0}J\left(r_1(0),r_3(1),y_1(0),y_3(1)\right)+{\beta }_1{t}_1(0)+{\beta }_2\left(t_1(1)-t_2(0)\right)+ \\ +{\beta }_3\left(t_2(1)-t_3(0)\right)+{\beta }_4\left(t_3(1)-T\right)+{\beta }_5\left(r_1(1)-r_2(0)\right)+{\beta }_6\left(r_2(1)-r_3(0)\right)+ \\ +{\beta }_7\left(y_1(1)-y_2(0)\right)+{\beta }_8\left(y_2(1)-y_3(0)\right)-{\alpha }_1{y}_2(0) \end{gathered}$$

$$\begin{gathered} \overline{\Pi }={\psi }_{r_1}{\rho }_{1}f(r_1,y_1,v_1)+{\psi }_{t_1}{\rho }_1+{\psi }_{y_1}{\rho }_{1}g(r_1,y_1,v_1)+ \\ +{\psi }_{r_2}{\rho }_{2}f(r_2,y_2,v_2)+{\psi }_{t_2}{\rho }_2+{\psi }_{y_2}{\rho }_{2}g(r_2,y_2,v_2)+ \\ +{\psi }_{r_3}{\rho }_{3}f(r_3,y_3,v_3)+{\psi }_{t_3}{\rho }_3+{\psi }_{y_3}{\rho }_{3}g(r_3,y_3,v_3)- \\ -\sigma {\rho }_{2}g(r_2,y_2,v_2)-h_1\phi \left(v_1\right)-h_3\phi \left(v_3\right), \end{gathered}$$

$$\begin{cases}\begin{aligned} -\frac{d\psi_{r_{1}}}{d\tau}&=\;\rho^{0}_{1}\Big{(}\psi_{r_{1}}f^{\prime}_{z}(r^{0}_{1},y^{0}_{1},v^{0}_{1})+\psi_{y_{1}}g^{\prime}_{z}\left(r^{0}_{1},y^{0}_{1},v^{0}_{1}\right)\Big{)},\\ -\frac{d\psi_{r_{2}}}{d\tau}&=\;\rho^{0}_{2}\Big{(}\psi_{r_{2}}f^{\prime}_{z}(r^{0}_{2},y^{0}_{2},v^{0}_{2})+(\psi_{y_{2}}-\sigma)g^{\prime}_{z}\left(r^{0}_{2},y^{0}_{2},v^{0}_{2}\right)\Big{)},\\ -\frac{d\psi_{r_{3}}}{d\tau}&=\;\rho^{0}_{3}\Big{(}\psi_{r_{3}}f^{\prime}_{z}(r^{0}_{3},y^{0}_{3},v^{0}_{3})+\psi_{y_{3}}g^{\prime}_{z}\left(r^{0}_{3},y^{0}_{3},v^{0}_{3}\right)\Big{)},\\[4.0pt] \psi_{r_{1}}(0)&=\alpha_{0}J^{\prime}_{z(0)}\,,\qquad\psi_{r_{1}}(1)=-\beta_{5},\\ \psi_{r_{2}}(0)&=-\beta_{5},\qquad\quad\;\;\psi_{r_{2}}(1)=-\beta_{6}\\ \psi_{r_{3}}(0)&=-\beta_{6},\qquad\quad\;\;\psi_{r_{3}}(1)=-\alpha_{0}J^{\prime}_{z(T)}\,,\end{aligned}\end{cases}$$

$$\begin{cases}\begin{aligned} -\frac{d\psi_{y_{1}}}{d\tau}&=\;\rho^{0}_{1}\Big{(}\psi_{r_{1}}f^{\prime}_{x}(r^{0}_{1},y^{0}_{1},v^{0}_{1})+\psi_{y_{1}}g^{\prime}_{x}\left(r^{0}_{1},y^{0}_{1},v^{0}_{1}\right)\Big{)},\\ -\frac{d\psi_{y_{2}}}{d\tau}&=\;\rho^{0}_{2}\Big{(}\psi_{r_{2}}f^{\prime}_{x}(r^{0}_{2},y^{0}_{2},v^{0}_{2})+(\psi_{y_{2}}-\sigma)g^{\prime}_{x}\left(r^{0}_{2},y^{0}_{2},v^{0}_{2}\right)\Big{)},\\ -\frac{d\psi_{y_{3}}}{d\tau}&=\;\rho^{0}_{3}\Big{(}\psi_{r_{3}}f^{\prime}_{x}(r^{0}_{3},y^{0}_{3},v^{0}_{3})+\psi_{y_{3}}g^{\prime}_{x}\left(r^{0}_{3},y^{0}_{3},v^{0}_{3}\right)\Big{)},\\[4.0pt] \psi_{y_{1}}(0)&=\alpha_{0}J^{\prime}_{x(0)}\,,\qquad\quad\psi_{y_{1}}(1)=-\beta_{7},\\ \psi_{y_{2}}(0)&=-\beta_{7}-\alpha_{1},\qquad\;\psi_{y_{2}}(1)=-\beta_{8},\\ \psi_{y_{3}}(0)&=-\beta_{8},\qquad\qquad\;\;\psi_{y_{3}}(1)=-\alpha_{0}J^{\prime}_{x(T)}\,,\\ \end{aligned}\end{cases}$$

$$\{\begin{array}{l}\begin{array}{rlrlrl}-\frac{d{\psi }_{t_1}}{d\tau }& =0,& {\psi }_{t_1}(0)& ={\beta }_1,& {\psi }_{t_1}(1)& =-{\beta }_2\\ -\frac{d{\psi }_{t_2}}{d\tau }& =0,& {\psi }_{t_2}(0)& =-{\beta }_2,& {\psi }_{t_2}(1)& =-{\beta }_3\\ -\frac{d{\psi }_{t_3}}{d\tau }& =0,& {\psi }_{t_3}(0)& =-{\beta }_3,& {\psi }_{t_3}(1)& =-{\beta }_4,\end{array}\end{array}$$

$$\{\begin{array}{l}\begin{array}{rl}{\overline{\Pi }}_{v_1}& =0,\\ {\overline{\Pi }}_{v_2}& =0,\\ {\overline{\Pi }}_{v_3}& =0,\end{array}\end{array}\iff \{\begin{array}{l}\begin{array}{rl}{\psi }_{r_1}{f}_u^{\prime }(r_1^0,y_1^0,v_1^0)+{\psi }_{y_1}{g}_u^{\prime }(r_1^0,y_1^0,v_1^0)& =\frac{h_1{\phi }_u^{\prime }(v_1^0)}{{\rho }_1^0},\\ {\psi }_{r_2}{f}_u^{\prime }(r_2^0,y_2^0,v_2^0)+{\psi }_{y_2}{g}_u^{\prime }(r_2^0,y_2^0,v_2^0)& =\sigma g_u^{\prime }(r_2^0,y_2^0,v_2^0),\\ {\psi }_{r_3}{f}_u^{\prime }(r_3^0,y_3^0,v_3^0)+{\psi }_{y_3}{g}_u^{\prime }(r_3^0,y_3^0,v_3^0)& =\frac{h_3{\phi }_u^{\prime }(v_3^0)}{{\rho }_3^0},\end{array}\end{array}$$

$$\{\begin{array}{l}\begin{array}{rl}{\overline{\Pi }}_{{\rho }_1}& =0,\\ {\overline{\Pi }}_{{\rho }_2}& =0,\\ {\overline{\Pi }}_{{\rho }_3}& =0,\end{array}\end{array}\iff \{\begin{array}{l}\begin{array}{rl}{\psi }_{r_1}f(r_1^0,y_1^0,v_1^0)+{\psi }_{y_1}{g}^{\prime }(r_1^0,y_1^0,v_1^0)+{\psi }_{t_1}& =0,\\ {\psi }_{r_2}f(r_2^0,y_2^0,v_2^0)+({\psi }_{y_2}-\sigma )g(r_2^0,y_2^0,v_2^0)+{\psi }_{t_2}& =0,\\ {\psi }_{r_3}f(r_3^0,y_3^0,v_3^0)+{\psi }_{y_3}g(r_3^0,y_3^0,v_3^0)+{\psi }_{t_3}& =0.\end{array}\end{array}$$

$$m(t):=\{\begin{array}{l}\begin{array}{rl}0& \text{mbox}on{\Delta }_1,\\ \sigma (\tau (t))& \text{mbox}on{\Delta }_2,\\ 0& \text{mbox}on{\Delta }_3,\end{array}\end{array}h(t):=\{\begin{array}{l}\begin{array}{rl}{h}_1(\tau (t))& \text{mbox}on{\Delta }_1,\\ 0& \text{mbox}on{\Delta }_2,\\ h_3(\tau (t))& \text{mbox}on{\Delta }_3.\end{array}\end{array}$$

$$\{\begin{array}{l}-{\dot{\psi }}_z=-\frac{1}{{\rho }_i^0}\frac{d{\psi }_{r_i}}{d\tau }={\psi }_z{f}_z^{\prime }+({\psi }_x-m)g_z^{\prime },\\ -{\dot{\psi }}_x=-\frac{1}{{\rho }_i^0}\frac{d{\psi }_{y_i}}{d\tau }={\psi }_z{f}_x^{\prime }+({\psi }_x-m)g_x^{\prime },\\ -{\dot{\psi }}_t=-\frac{1}{{\rho }_i^0}\frac{d{\psi }_{t_i}}{d\tau }=0.\end{array}$$

$$\{\begin{array}{l}\begin{array}{rlrl}{\psi }_x(t_1^0-0)& =-{\beta }_7,& {\psi }_x(t_1^0+0)& =-{\beta }_7-{\alpha }_1,\\ {\psi }_x(t_2^0-0)& =-{\beta }_8,& {\psi }_x(t_2^0+0)& =-{\beta }_8,\end{array}\end{array}$$

Full text

1 Introduction

It is a well-known fact that optimality conditions in problems with state constraints are difficult for application in view of a nonstandard character of the state constraint multiplier. In their seminal work [1], A.Ya. Dubovitskii and A.A. Milyutin suggested to take this multiplier in the form of non-negative measure concentrated on the boundary set of the optimal trajectory (see also later works [2, 3]). This corresponds to the functional meaning of the state constraint, but then the adjoint equation contains a measure (more precisely, its generalized derivative 111 For a function $\mu(t)$ of bounded variation, its generalized derivative $\dot{\mu}(t)=d\mu(t)/dt$ is a generalized function in the sense that $\dot{\mu}(t)\,dt=d\mu(t)$ is the Riemann–Stieltjes measure generated by the function $\mu(t).$ If $\mu(t)$ is absolute continuous, then $\dot{\mu}(t)$ is a usual Lebesgue integrable function;  if $\mu(t)$ is discontinuous at a point $t_{*}\,,$ then $\dot{\mu}(t)$ contains the Dirac $\delta-$function at $t_{*}\,.$); hence, one comes to a differential equation of a new, yet uninvestigated type. Therefore, from the very beginning of studying such problems, many specialists tried to avoid somehow this difficulty in order to keep the adjoint equation as an ODE of convenient type.

If the boundary set of the trajectory is a segment, one can differentiate the state constraint and reduce it to a mixed control-state constraint, for which the stationarity conditions can be formulated with the usage of standard objects. The result can be then represented in terms of the original problem. This way was firstly suggested by R.V.Gamkrelidze in the classical book [4], earlier than paper [1], but its realization involves a nontrivial further step:  one has to obtain the non-negativity of the measure (the state constraint multiplier), including the sign of the atoms of measure at junction points, which was not completely done in [4].

Thus, for the problems with state constraints there are two forms of optimality conditions (say, the maximum principle):  the form of Gamkrelidze and the form of Dubovitskii–Milyutin. A natural question is how these two forms are connected. In paper [5] and then in [6], it was shown, by a simple change of the adjoint variable 222If $\psi(t)$ is the adjoint variable in the Dubovitskii–Milyutin form, $\Phi(t,x(t))\leq 0$ is the state constraint, and a monotone function $\mu(t)$ generates the corresponding measure, then $\widetilde{\psi}(t)=\psi(t)-\mu(t)\,\Phi^{\prime}_{x}(t,x^{0}(t))$ is the adjoint variable in the Gamkrelidze form., that one can pass from the conditions in the Dubovitskii–Milyutin form to the conditions in the form of Gamkrelidze, but the possibility of the inverse passage was not investigated.

In this paper, we consider a special class of problems and reference trajectories, in which the connection between the non-negativity of the measure and the minimization of the cost is the most transparent. In this class, one can completely fulfill Gamkrelidze’s idea and prove the non-negativity of the measure, thus showing that Gamkrelidze’s approach allows to obtain the conditions in Dubovitskii–Milyutin’s form. For simplicity, here we consider only necessary conditions of the so-called extended weak minimality (i.e., stationarity conditions), leaving the question about conditions of the strong minimality (the maximum principle) for further investigations.

2 Problem Statement

On a fixed time interval, consider the following optimal control problem with a state constraint:

[TABLE]

Here, $z\in\mathbb{R}^{n}$ and $x\in\mathbb{R}^{1}$ are state variables, $u\in\mathbb{R}^{m}$ is a control, the functions $z(\cdot)$ and $x(\cdot)$ are absolute continuous, $u(\cdot)$ is measurable and bounded. We will assume that the functions $f,\,g,\,\varphi$ of dimensions $n,\,1,\,\mbox{d}(\varphi),$ respectively, are defined and continuous on an open subset $\mathcal{Q}\subset\mathbb{R}^{n+1+m}$ together with their first-order partial derivatives w.r.t $z,x,u.$ (The function $\varphi(u)$ can be formally considered as a function of variables $z,x,u$). Note that the state constraint is imposed only on the scalar state coordinate $x,$ so it has the simplest form $x\geq 0.$

Definition 2.1**.**

A triple of functions $w=(z,x,u)$ of the corresponding functional classes defined on $[0,T]$ and satisfying equations $\dot{z}=f(z,x,u),$ $\dot{x}=g(z,x,u)$ is called a process of problem A. A process is called admissible if it satisfies all the constraints of the problem.

3 The Reference Trajectory

Consider a reference process $w^{0}=(z^{0},x^{0},u^{0})$ such that the trajectory $x^{0}(t)$ touches the state boundary only on a segment $[t^{0}_{1},t^{0}_{2}],$ where $0<t^{0}_{1}<t^{0}_{2}<T.$ In other words, the interval $\Delta:=[0,T]$ is divided into parts $\Delta_{1}:=[0,t^{0}_{1}],$ $\Delta_{2}:=[t^{0}_{1},t^{0}_{2}],$ and $\Delta_{3}:=[t^{0}_{2},T]$ such that $x^{0}(t)>0$ on $[0,t^{0}_{1}),$ $x^{0}(t)=0$ on $\Delta_{2},$ and $x^{0}(t)>0$ on $(t^{0}_{2},T].$ In addition, we suppose the control $u^{0}$ to be continuous on $\Delta_{1}\,,\Delta_{3}$ and Lipschitz continuous on $\Delta_{2}$ (for convenience, we assume that the function $u^{0}$ at time moments $t^{0}_{1},\,t^{0}_{2}$ has both left and right values), moreover, $\varphi_{s}(u^{0}(t))<0$ on $\Delta_{2}$ for all $s,$ and the following strict inequalities hold at the moments $t^{0}_{1},\,t^{0}_{2}$:

[TABLE]

which mean that the landing to the state boundary and the leaving it occurs with nonzero time derivatives. We also suppose that $g^{\prime}_{u}(z^{0}(t),x^{0}(t),u^{0}(t))\neq 0$ on the boundary arc $\Delta_{2}\,,$ i.e., that the state constraint is of order 1, and the gradients $\varphi^{\prime}_{s}(u^{0}(t)),\;\,s\in I(u^{0}(t)),$ are positive independent for all $t\in\Delta_{1}\cup\Delta_{3}\,$ (i.e., their nontrivial linear combination with non-negative coefficients cannot vanish). Here $I(u)=\{s\,:\;\varphi_{s}(u)=0\}$ is the set of active indices.

For short, we will write the control constraints in the vector form $\varphi(u)\leq 0.$

Throughout this paper, we assume that the above assumptions are satisfied for problem A.

Note that these assumptions are not easily verifiable a priori;  however, they are often satisfied in typical real problems. As any other a priori assumptions, they can be considered, together with necessary conditions of optimality, as a united collection of conditions for the search of optimal trajectories. In the book [4], a less restrictive assumption on the reference trajectory $x^{0}(t)$ is imposed:  it may touch the state boundary not on one segment, but on a finite number of segments. The reference control $u^{0}(t)$ is not assumed in [4] to lie in the interior of the set $\varphi(u)\leq 0$ on $\Delta_{2}\,;$ instead, it is assumed that the gradient $g^{\prime}_{u}(z^{0}(t),x^{0}(t),u^{0}(t))$ together with the active gradients $\varphi^{\prime}_{s}(u^{0}(t))$ are linearly independent on $\Delta_{2}\,.$ We do not consider here these more complicated cases in order to avoid more cumbersome technicalities, which would distract the reader’s attention from the main line of argumentation.

4 The Type of Minimum

We admit not only uniformly small variations of the control, but also small variations of its discontinuity points. This corresponds to consideration of the “extended” weak minimality. Recall its definition (see, e.g. [7]) for a problem of type A.

Definition 4.1**.**

An admissible process $w^{0}(t)=(z^{0}(t),x^{0}(t),u^{0}(t))$ provides the extended weak minimumality in problem A  if there exists an $\varepsilon>0$ such that, for any Lipschitz continuous surjective mapping $\sigma:[0,T]\to[0,T]\,$ satisfying $|\sigma(t)-t|<\varepsilon$ and $|\dot{\sigma}(t)-1|<\varepsilon,$ and for any admissible process $w(t)=(z(t),x(t),u(t))$ satisfying the conditions

[TABLE]

one has $J(w)\geq J(w^{0}).$

The conditions on $\sigma$ imply $\sigma(0)=0$ and $\sigma(T)=T.$ If we take $\sigma(t)=t,$ then relations (3) describe the usual uniform closeness between the processes $w^{0}$ and $w$ both in the state and control variables. However, for an arbitrary $\sigma(t),$ relations (3) extend the set of ”competing”  processes, and thus the extended weak minimality is stronger than the classical weak minimality. The choice of arbitrary $\sigma(t)$ close to $\hat{\sigma}(t)=t$ corresponds to a variation (deformation) of the current time within the interval $[0,T]$ in addition to the usual uniformly small variations of $z(t),\,x(t)$ and $u(t)$ for the fixed values of $t.$

If the control $u^{0}(t)$ is continuous, the notion of extended weak minimality reduces to the usual notion of weak minimality. However, in the case of discontinuous $u^{0}(t),$ the usual small variations of the control (corresponding to the weak minimality) leave the points of discontinuity of $u^{0}(t)$ invariable, whereas the extended weak minimality allows for small variations of them.

5 Passage from Problem A to a Problem with Mixed Control-State Constraints

Following [8], we introduce a new time variable $\tau\in[0,1]$ and consider the initial time variable $t$ on each segment $\Delta_{i}$ as a new state variable $t_{i}(\tau)$ subject to equation $\dfrac{dt_{i}}{d\tau}=\rho_{i}(\tau),$ where the functions $\rho_{i}(\tau)>0,$ $i=1,2,3$ are additional controls.

On the segment $[0,1],$ introduce the state variables $r_{i}(\tau)=z(t_{i}(\tau)),$ $y_{i}(\tau)=x(t_{i}(\tau)),$ and the controls $v_{i}(\tau)=u(t_{i}(\tau)).$ Hence, the following equations are satisfied:

[TABLE]

Thus, we “replicate” the variables of the original problem by taking their reductions to the intervals $\Delta_{i}$ and considering all of these reductions as new variables of the new time 333This natural trick of replication of variables was first proposed, probably, in [9], and later was also used, may be independently, by many authors, e.g. in [10, 11, 12, 13, 8, 14, 15].. In terms of these new variables, we now formulate a new problem related to our problem A.

Since the original state variables $z,\,x$ are continuous at times $t_{1},\,t_{2}$ (close to $t^{0}_{1},\,t^{0}_{2}),$ the new state variables should satisfy the junction conditions

[TABLE]

Moreover, since the time interval $[0,T]$ is fixed, the variables $t_{i}$ should satisfy the boundary conditions $\;t_{1}(0)=0$ and $t_{3}(1)-T=0.$

Instead of state constraint $y_{2}(\tau)\geq 0$ on $[0,1],$ we will consider the following pair of an endpoint and a mixed control-state constraints:

[TABLE]

while the control constraints will be now written in the form

[TABLE]

In the new problem, we will consider the “classical” weak minimality. Therefore, we do not need to consider the open constraints $\rho_{i}>0$ as well as the constraint $\varphi(v_{2}(\tau))\leq 0,$ since under our assumptions the control $v_{2}^{0}(\tau)$ lies strictly in its interior.

Thus, we come to the following optimal control problem on the time interval $\tau\in[0,1]:$

[TABLE]

under the following constraints:

[TABLE]

[TABLE]

[TABLE]

[TABLE]

This problem will be called problem B.  Here, $\rho_{i},v_{i}$ are the controls and $r_{i},\,y_{i},\,t_{i}$ the state variables, $i=1,2,3.$ Note that constraints (5) (included in (8) and (10)) define a smaller class of admissible trajectories than the state constraint $y_{2}(\tau)\geq 0$ does, so the new problem is not equivalent to the initial problem A.  Later, in Sec. 8, we will also take into account nonconstant variations of $y_{2}(\tau),$ i.e., of $x(t)$ on the boundary interval.  On the other hand, the new problem does not involve the state constraints $y_{1}\geq 0$ and $y_{3}\geq 0,$ so it allows for a bigger class of admissible trajectories.

It is easy to see that, to each admissible process $w=(z,x,u)$ of problem A with $x(t)=\mbox{const}\,$ on an interval $[t_{1},t_{2}],$ one can associate a (not unique) admissible process $\gamma=(r_{i},y_{i},t_{i},\rho_{i},v_{i})$ of problem B (by choosing, e.g. $\rho_{i}(\tau)\equiv|\Delta_{i}|$), and to each admissible process of problem B one can associate, simply by setting $\tau=\tau(t),$ a unique admissible process of problem A  with $x(t)=\mbox{const}\,$ on $[t_{1},t_{2}].$

Let us establish a relation between the extended weak minimality in problem A and the “classical” weak minimality in problem B.

Lemma 5.1**.**

Let the process $w^{0}=(z^{0}(t),x^{0}(t),u^{0}(t))$ with the boundary arc $[t_{1}^{0},t_{2}^{0}]$ provide the extended weak minimality in problem A.  Then the corresponding process $\gamma^{0}=(r^{0}_{i}(\tau),\,y^{0}_{i}(\tau),\,t^{0}_{i}(\tau),\,\rho^{0}_{i}(\tau),\,v^{0}_{i}(\tau),$ $i=1,2,3)$ provides the weak minimality in problem B.

Proof.

Suppose that the process $\gamma^{0}$ does not provide the weak minimality in problem B. Then, there exists a sequence of uniformly convergent processes $\gamma\rightrightarrows\gamma^{0}$ of problem B, such that $J_{B}(\gamma)<J_{B}(\gamma^{0}).$ According to (2), there exist such $\theta<1$ and $c>0$ that

[TABLE]

Then, for sufficiently far members of the sequence, we get

[TABLE]

From here with account of $y_{1}(1)\geq 0$, we get $y_{1}(\tau)>0$ on $[\theta,1).$ Consider the segment $[0,\theta].$ Here $y_{1}^{0}(\tau)>0,$ hence $y_{1}^{0}(\tau)\geq b$ for some $b>0.$ Therefore, $y_{1}(\tau)\geq b/2>0$ for sufficiently far members of the sequence. Thus, $y_{1}(\tau)>0$ on the whole semi-open interval $[0,1).$ Similarly, one can prove that $y_{3}(\tau)>0$ on the whole semi-open interval $(0,1].$ The inequality $y_{2}(\tau)\geq 0$ obviously holds on $[0,1],$ since $y_{2}=\mbox{const}\,$ and $y_{2}(0)\geq 0.$

Thus, for the corresponding processes $w=(z,x,u)$ of problem A, we get

[TABLE]

where $\,t_{1}\to t_{1}^{0},\;\;t_{2}\to t_{2}^{0}\,.$ The constraints $\varphi(u(t))\leq 0$ are satisfied in view of inequalities $\varphi(v_{i}(\tau))\leq 0,$ $i=1,2,3.$

So, the prelimiting processes $w$ are admissible in problem A with the cost $J_{A}(w)=J_{B}(\gamma)<J_{B}(\gamma^{0})=J_{A}(w^{0}),$ a contradiction with the extended weak minimality in problem A  at the process $w^{0}.$ ∎∎

6 Stationarity Conditions for Problem B

Let us agree to denote the derivatives of $f,\,g$ w.r.t. first, second, and third arguments as $f^{\prime}_{z},\,g^{\prime}_{z},$ $f^{\prime}_{x},\,g^{\prime}_{x},$ $f^{\prime}_{u},\,g^{\prime}_{u},$ respectively, no matter on which variables these functions depend.

The three constraints (10) will be treated as mixed control-state ones. In order to apply the known stationarity conditions, we have to check whether these constraints are regular along the reference process $\gamma^{0}(\tau).$

According to [16, 17, 18], mixed control-state constraints $\Phi_{i}(t,x,u)\leq 0$ and $G_{j}(t,x,u)=0$ of equality and inequality type given by smooth functions on $\mathbb{R}\times\mathbb{R}^{n}\times\mathbb{R}^{r}$ are called regular  at a point $(t,x,u)$ if their gradients w.r.t control are positive–linearly independent, which means that there do not exist multipliers $\alpha_{i}\geq 0$ and $\beta_{j}$ with $\sum\alpha_{i}+\sum|\beta_{j}|>0$ and $\alpha_{i}\,\Phi_{i}(t,x,u)=0$ such that

[TABLE]

Applying this to the constraints (10), one can easily see that the gradients w.r.t control $v=(v_{1},v_{2},v_{3})$ of these constraints are positive–linearly independent along the reference process (since they decompose into the gradients w.r.t each component $v_{i}),$ hence their gradients w.r.t the “full” control vector $(v,\rho)$ are the more so positive–linearly independent, and thus, the mixed constraints in problem B  are regular.

Assume the process $\gamma^{0}$:= $(r_{i}^{0}(\tau),\,y_{i}^{0}(\tau),\,t_{i}^{0}(\tau),\,\rho_{i}^{0}(\tau),\,v_{i}^{0}(\tau),$ $i=1,2,3)$ provides the weak minimality in problem B. Then it satisfies the stationarity conditions, which say the following (see, e.g. [16, 17, 18]):  there exist multipliers $\alpha_{0},\,\alpha_{1},$ $\beta_{j},\;j=1,...,8,$ Lipschitz functions $\psi_{r_{i}},\psi_{y_{i}},\psi_{t_{i}},$ $i=1,2,3,$ measurable bounded functions $h_{1}(\tau),\,h_{3}(\tau)$ of dimension $d(\varphi),$ and a measurable bounded scalar function $\sigma(\tau),$ such that the following conditions are satisfied:

nontriviality condition

[TABLE]

non-negativity condition

[TABLE]

complementary slackness condition

[TABLE]

and such that, in terms of the endpoint Lagrange function

[TABLE]

and the extended Pontryagin function

[TABLE]

the following conditions are also satisfied:

adjoint equations and transversality conditions

[TABLE]

[TABLE]

[TABLE]

stationarity conditions w.r.t  controls $v_{i},$ $i=1,2,3:$

[TABLE]

and stationarity conditions w.r.t  controls $\rho_{i},$ $i=1,2,3:$

[TABLE]

Here, $J^{\prime}_{z(0)},\,J^{\prime}_{z(T)},\,J^{\prime}_{x(0)},\,J^{\prime}_{x(T)}$ are the derivatives of $J(z(0),z(T),x(0),x(T))$ w.r.t the corresponding variables, taken at the point $(r_{1}^{0}(0),r_{3}^{0}(1),y_{1}^{0}(0),y_{3}^{0}(1)).$

Note that, since the function $u^{0}(\tau)$ is Lipschitz continuous on $\Delta_{2}\,,$ the second equation in (19) and the nondegeneracy of $g^{\prime}_{u}$ implies that $\sigma(\tau)$ is also Lipschitz continuous.

Fist of all, let us state the following

Lemma 6.1**.**

$\alpha_{0}>0$* (hence, one can set $\alpha_{0}=1$).*

Proof.

Suppose that $\alpha_{0}=0.$ Then by (16)–(17), the pair $(\psi_{r_{1}},\,\psi_{y_{1}})$ satisfies a linear system of ODEs with initial conditions $\psi_{r_{1}}(0)=0,\;\psi_{y_{1}}(0)=0,$ whence $\psi_{r_{1}}$ and $\psi_{y_{1}}$ identically vanish. Similarly, $\psi_{r_{3}}$ and $\psi_{y_{3}}$ vanish too, hence $\beta_{5}=\beta_{6}=0$ and $\beta_{7}=\beta_{8}=0.$

In view of (19), we get $h_{1}(\tau)\equiv h_{3}(\tau)\equiv 0$ and $\sigma(\tau)=A\psi_{r_{2}}+B\psi_{y_{2}}$ with some Lipschitz continuous functions $A(\tau),\,B(\tau);$ moreover, since $\psi_{r_{2}}(1)=0$ and $\psi_{y_{2}}(1)=0,$ we have $\sigma(1)=0.$ Thus, in view of (16)–(17), $\psi_{r_{2}}$ and $\psi_{y_{2}}$ satisfy a system of linear ODEs with zero boundary values at $\tau=1,$ which implies that $\psi_{r_{2}}\equiv 0$ and $\psi_{y_{2}}\equiv 0.$ Therefore, $\sigma(\tau)\equiv 0$ and by (17) $\alpha_{1}=0,$ then in view of (20) we get $\psi_{t_{1}}=\psi_{t_{2}}=\psi_{t_{3}}=0,$ hence $\beta_{1}=\beta_{2}=\beta_{3}=\beta_{4}=0.$ Thus, the whole collection of multipliers is trivial, a contradiction with (11). ∎∎

7 Stationarity Conditions in Terms of the Original Problem A

Let us rewrite the stationarity conditions from Sec. 6 in terms of the original problem (1). To do this, define functions $m(t)$ and $h(t)$ on the interval $[0,T]$ as follows:

[TABLE]

Notice that function $m(t)$ is Lipschitz continuous on the intervals $\Delta_{1},\,\Delta_{2},$ $\Delta_{3}.$ By $\dot{m}(t)$ we will denote its generalized derivative. Since \dfrac{d\psi}{dt}=\dfrac{d\psi}{d\tau}\Big{/}\dfrac{dt}{d\tau}, then, getting back from the new time $\tau$ to the original time $t$ in equations (16)–(18), we obtain the following equations on the whole interval $[0,T]$:

[TABLE]

Since the state variables $r_{i},y_{i},t_{i}$ of problem B  are continuously joined at the corresponding ends of interval $[0,1]$ by the junction conditions (4), the state variables $z(t),\,x(t)$ of problem A are continuous (and moreover, Lipschitz continuous). By similar arguments, the adjoint variables $\psi_{z},\;\psi_{t}$ of problem A are also Lipschitz continuous.  Consider the function $\psi_{x}$.

Note first that it is Lipschitz continuous on every interval $\Delta_{i},$ $i=1,2,3$. Rewriting the transversality conditions for $\psi_{x}$ in terms of problem A, we get the following junction conditions:

[TABLE]

i.e., $\psi_{x}$ is continuous at $t_{2}^{0}$ and has the jump $\Delta\psi_{x}(t_{1}^{0})=-\alpha_{1}\leq 0$ at the point $t_{1}.$ At the ends of interval $[0,T],$ it satisfies the transversality conditions

[TABLE]

If we introduce the extended Pontryagin function for the problem with mixed control-state constrains

[TABLE]

then, in view of (22), we obtain the fulfilment of adjoint equations

[TABLE]

on the interval $[0,T]$ except the points $t_{1}^{0},\,t_{2}^{0}\,,$ and the fulfilment of stationarity condition w.r.t. control $u$ for all $t$:

[TABLE]

Let us now rewrite these conditions in terms of problem A involving a state constraint. To do this, set $\widetilde{\psi}_{x}(t)=\psi_{x}(t)-m(t),$ introduce the Pontryagin function of this problem

[TABLE]

and the extended Pontryagin function

[TABLE]

with a multiplier $\dot{m}(t)$ at the state constraint. It is easy to verify that, along the interval $[0,T]$ except the points $t_{1}^{0},\,t_{2}^{0}\,,$ the adjoint equations

[TABLE]

and stationarity condition w.r.t. control $\overline{H}^{\prime}_{u}=0$ hold.

The transversality conditions (24) are obviously still satisfied. In view of (21) and (23), the adjoint variable $\widetilde{\psi}_{x}$ has the jumps

[TABLE]

Since $\dot{\psi}_{t}=0$, equation (20) rewritten in time $t$ turns into

[TABLE]

which is equivalent to the “energy conservation law” $H(z^{0},x^{0},u^{0})=\mbox{const}\,.$

Note that we get $x^{0}=0$ on $\Delta_{2}\,,$ while outside $\Delta_{2}$ we get $\dot{m}\equiv 0,$ i.e., the complementary slackness condition for the state constraint holds:

[TABLE]

The definition of ${h}$ and condition (13) imply that the complementary slackness condition holds also for the control constraint:

[TABLE]

8 Non-negativity of Multiplier at the State Constraint

We have obtained stationarity conditions in problem A, in which the measure is absolute continuous on the interval $\Delta_{2}$ with density $\dot{m}(t)$ and has the jumps (atoms) $-\alpha_{1}-{m}(t_{1}^{0}+0)$ and ${m}(t_{2}^{0}-0)$ at the points $t_{1}^{0},\,t_{2}^{0},$ respectively. Our next aim is to define the sign of its density and jumps. To this end, we take into account that we have feasible variations $\bar{x}(t)\geq 0$ on $\Delta_{2}$ in our disposal.

Consider first any triple $\bar{w}(t)=(\bar{z}(t),\,\bar{x}(t),\,\bar{u}(t))$ satisfying the linearized system in variations along the process $w^{0}(t)$ on $[0,T]$:

[TABLE]

The main technical formula to use is defined by the following

Lemma 8.1**.**

Let be given Lipschitz continuous functions $\psi_{z}(t),\,z(t),\,x(t)$ and measurable bounded functions $h(t),\,u(t)$ on an interval $[0,T].$ Let be also given functions $\psi_{x}(t),\,m(t)$ Lipschitz continuous on intervals $\Delta_{1}=[0,t_{1}],$ $\Delta_{2}=[t_{1},t_{2}],$ $\Delta_{3}=[t_{2},T]$ with possible jumps at the points $t_{1},\,t_{2},$ where $0<t_{1}<t_{2}<T,$ such that the following relations hold on every above interval:

[TABLE]

Then any solution $\bar{w}=(\bar{z},\bar{x},\bar{u})$ of system (34) on $[0,T]$ satisfies the following equality:

[TABLE]

where $\Delta\psi_{x}(t_{i})$ are the jumps of $\psi_{x}$ at the points $t_{1},\,t_{2}\,.$

Proof.

In view of (35), we have, on every interval $\Delta_{i}$:

[TABLE]

Integrating this equality on the whole interval $[0,T]$ (on $\Delta_{2}\,,$ we integrate $m\dot{\bar{x}}$ by parts)  and taking into account possible jumps of $\psi_{x}$ at the points $t_{1},\,t_{2},$ we get that the left hand part of (36) is equal to

[TABLE]

which implies the required equality (36). ∎∎

Now, we introduce variations of some special type.

Lemma 8.2**.**

For any Lipschitz continuous function $\varkappa(t)$ defined on the interval $\Delta_{2}=[t_{1}^{0},\,t_{2}^{0}],$ there exists a solution $(\bar{z}(t),\bar{x}(t),\bar{u}(t))$ of system (34) on $\Delta_{2}$ such that $\bar{x}(t)=\varkappa(t).$

Proof.

Let us set $\bar{x}(t)=\varkappa(t),\,$ $\bar{u}(t)=v(t)\,g^{\prime}_{u}\,,$ where $v(t)$ is a scalar function to be found. Since $g^{\prime}_{u}(z^{0},x^{0},u^{0})\neq 0,$ from the second equation of system (34) we obtain v(t)=\,\big{(}\dot{\varkappa}-g^{\prime}_{z}\bar{z}-g^{\prime}_{x}\varkappa\big{)}/\,|g^{\prime}_{u}|^{2}\,. Substituting the corresponding $\bar{u}(t)$ into the first equation of system (34), we come to the following nonhomogeneous equation with respect to $\bar{z}:$

[TABLE]

Setting for definiteness $\bar{z}(t_{1})=0,$ we get the solution of this equation, and then define $v(t)$ and $\bar{u}(t).$ ∎∎

Consider now any $\varkappa(t)>0$ on $\Delta_{2}=[t_{1}^{0},\,t_{2}^{0}].$ By Lemma 8.2, the system (34) has a solution $\bar{w}(t)=(\bar{z}(t),\bar{x}(t),\bar{u}(t))$ on $\Delta_{2}$ with $\bar{x}(t)=\varkappa(t).$ To construct the corresponding process, which will be compared with the optimal one $w^{0},$ we have to go back to the original nonlinear system $\dot{z}=f(z,x,u),$   $\dot{x}=g(z,x,u).$ Note that (34) is the variational system for the latter one. According to the main property of variational equation, for any $\varepsilon>0$ there exists a correction $\tilde{w}_{\varepsilon}=(\tilde{z}_{\varepsilon},\tilde{x}_{e},\tilde{u}_{e})$ with $||\tilde{w}_{\varepsilon}||_{\infty}\leq o(\varepsilon)$ as $\varepsilon\to 0+$ such that the triple $w_{\varepsilon}=w^{0}+\varepsilon\bar{w}+\tilde{w}_{\varepsilon}$ satisfies the original system on $\Delta_{2}.$ It is easy to verify that this triple satisfies also conditions $x_{\varepsilon}(t)>0$ and $\varphi(u_{\varepsilon})<0$ on $\Delta_{2}\,.$

Now, let us extend this triple, defined only on $\Delta_{2}\,,$ to a process defined on the whole interval $[0,T].$ To do this, on $\Delta_{1}=[0,t_{1}^{0}]$ we set $u_{\varepsilon}=u^{0}$ (i.e., $\bar{u}=0$) and solve the nonlinear system with initial conditions $z_{\varepsilon}(t_{1}^{0}),\;x_{\varepsilon}(t_{1}^{0}).$ On $\Delta_{3}\,,$ we again set $u_{\varepsilon}=u^{0}\;(\bar{u}=0)$ and solve the nonlinear system with the initial conditions $z_{\varepsilon}(t_{2}^{0}),\;x_{\varepsilon}(t_{2}^{0}).$ Thus, we get a process $w_{\varepsilon}=(z_{\varepsilon},\,x_{\varepsilon},\,u_{\varepsilon})$ on the whole interval $[0,T]$ that by definition satisfies the constraint $\varphi(u_{\varepsilon})\leq 0.\;$

Note that $\dfrac{dw_{\varepsilon}(t)}{d\varepsilon}=\bar{w}(t)=(\bar{z},\bar{x},\bar{u}),$ where $\bar{u}=0$ on $\Delta_{1}\cup\Delta_{3}\,$ and $\bar{u}$ on $\Delta_{2}$ is the above function from Lemma 8.2, satisfies the linear system (34) on $[0,T].$ In particular, the pair $\left(\bar{z}=\dfrac{dz_{\varepsilon}}{d\varepsilon},\;\,\bar{x}=\dfrac{dx_{\varepsilon}}{d\varepsilon}\right)$ satisfies on $\Delta_{1}\cup\Delta_{3}$ the system of linear equations in variations

[TABLE]

Lemma 8.3**.**

$x_{\varepsilon}(t)>0$* on $\Delta_{1}\cup\Delta_{3}$ for small $\varepsilon>0,$ except the points $t_{1}^{0},\,t_{2}^{0}\,.$*

Proof.

Define $\zeta(t)=\bar{z}(t)$ on $\Delta_{2}$ and consider the interval $\Delta_{3}.$ On this interval, the pair $(z_{\varepsilon},x_{\varepsilon})$ satisfies the same nonlinear system as the pair $(z^{0},x^{0}),$ but with the corrected initial conditions $z_{\varepsilon}(t_{2}^{0}),\,x_{\varepsilon}(t_{2}^{0}).$ Then, the pair $(\bar{z},\bar{x})$ satisfies the linear system (37) with initial conditions $\bar{z}(t_{2}^{0})=\zeta(t_{2}^{0}),$ $\bar{x}(t_{2}^{0})=\varkappa(t_{2}^{0}).$ Since $c:=\varkappa(t_{2}^{0})>0,$ there exists such $\delta>0$ that $\bar{x}(t)\geq c/2$ on $[t_{2}^{0},\,t_{2}^{0}+\delta].$ Then $x_{\varepsilon}(t)\geq\varepsilon c/3$ on this interval for small enough $\varepsilon>0.$

Since $x^{0}(t)\geq\mbox{const}\,>0$ on $[t_{2}^{0}+\delta,T],$ we get $x_{\varepsilon}(t)>0$ for small $\varepsilon>0.$ Thus, $x_{\varepsilon}(t)>0$ on the whole $\Delta_{3}\setminus\{t_{2}^{0}\}.$ The interval $\Delta_{1}$ is considered similarly. ∎∎

Thus, the constructed process $w_{\varepsilon}$ satisfies all the constraints of problem (1) and, since the process $w^{0}$ provides the weak minimality, we have

[TABLE]

Let us apply Lemma 8.1 to the constructed triple $(\bar{z},\bar{x},\bar{u})$ and functions $\psi_{x},\,m,\,h$ defined in (21)–(24).  Since $m=0$ on $\Delta_{1}\cup\Delta_{3}\,,$ the first two integrals in (36) disappear, and since $h=0$ on $\Delta_{2}$ and $\bar{u}=0$ on $\Delta_{1}\cup\Delta_{3}\,,$ the last integral disappears too. According to transversality conditions (24), the left hand part of relation (36) is exactly

[TABLE]

hence

[TABLE]

This inequality holds for any Lipschitz continuous function $\bar{x}(t)=\varkappa(t)>0$ on $\Delta_{2}.$ Now, take any $\varkappa(t)\geq 0$ on $\Delta_{2}\,.$ Approximating it uniformly by functions $\varkappa(t)>0$ and passing to the limit in (39), we obtain that inequality (39) holds for any Lipschitz continuous function $\varkappa(t)\geq 0$ on $\Delta_{2}\,.$ Considering only $\varkappa(t)$ with zero values at the endpoints of $\Delta_{2},$ we get $\int_{\Delta_{2}}\dot{m}\,\bar{x}\,dt\geq 0,$ which implies $\dot{m}(t)\geq 0$ almost everywhere on $\Delta_{2}\,.$

Consider now functions $\varkappa(t)\geq 0$ that vanish on $[t_{1}^{0}+\delta,\,t_{2}^{0}]$ for a small $\delta>0$ and satisfy $\varkappa(t)\leq 1$ with $\varkappa(t_{1}^{0})=1.$ If $\delta\to 0+\,,$ the integral in the right hand side of (39) tends to zero, thus $-\Delta\psi_{x}(t_{1}^{0})+m(t_{1}^{0}+0)\geq 0.$ In view of equality $\Delta\psi_{x}(t_{1}^{0})=-\alpha_{1},$ we get $\alpha_{1}+m(t_{1}^{0}+0)\geq 0.$ Similarly, we get $-m(t_{2}^{0}-0)\geq 0$ in view of continuity of $\psi_{x}$ at the point $t_{2}^{0}\,.$

Thus, we have proved the following

Lemma 8.4**.**

Let the process $w^{0}$ provide the extended weak minimality in problem A. Then $\dot{m}(t)\geq 0$ on $\Delta_{2}$ (i.e., $m(t)$ decreases on $\Delta_{2});$ moreover, $\alpha_{1}+m(t_{1}^{0}+0)\geq 0$ and $-m(t_{2}^{0}-0)\geq 0.$

Let us get back to the function $\widetilde{\psi}_{x}=\psi_{x}-m$ having the jumps (30).

To “equalize” these jumps, we introduce the function

[TABLE]

Then, according to Lemma 8.4,

[TABLE]

and $\dot{\mu}(t)\geq 0$ for $t\neq t_{1}^{0},\;\,t\neq t_{2}^{0}.$ The jumps of adjoint variable $\widetilde{\psi}_{x}$ at junction points have now a “symmetric” form: $\;\Delta\widetilde{\psi}_{x}(t_{i}^{0})=\,-\Delta\mu(t_{i}^{0}),$ $i=1,2.$ The adjoint equation for $\widetilde{\psi}_{x}$ (see (29) now looks as follows:

[TABLE]

where $\dot{\mu}$ is the derivative in the sense of generalized functions. This equation should be regarded as an equality between measures:

[TABLE]

9 The Final Result

We now summarize our findings:

Theorem 9.1**.**

Let $w^{0}(t)=\left(z^{0}(t),x^{0}(t),u^{0}(t)\right)$ be an admissible process in problem A such that $x(t)=0$ on $\Delta_{2}=[t^{0}_{1},t^{0}_{2}],$ $x(t)>0$ on $[0,T]\setminus\Delta_{2},$ $\varphi_{i}(u^{0}(t))<0$ on $\Delta_{2}\,,$ assumption (2) holds, and let this process provide the extended weak minimality. Then there exist a Lipshitz continuous function $\psi_{z}(t),$ a constant $c,$ functions $\widetilde{\psi}_{x}(t)$ and $\mu(t)$ Lipschitz continuous on each interval $\Delta_{i},\;\,i=1,2,3,$ with possible jumps at $t^{0}_{1},\,t^{0}_{2}\,,$ and a measurable bounded function $h(t),$ which generate the Pontryagin function

[TABLE]

and the extended Pontryagin function

[TABLE]

such that the following conditions hold:

1. (a)

non-negativity conditions

[TABLE] 2. (b)

complementary slackness

[TABLE] 3. (c)

adjoint equations

[TABLE] 4. (d)

transversality conditions

[TABLE] 5. (e)

jumps conditions for the adjoint variable $\widetilde{\psi}_{x}$

[TABLE] 6. (f)

the energy conservation law

[TABLE] 7. (g)

stationarity condition w.r.t. control

[TABLE]

Remark 9.1**.**

Note again that theorem 9.1 is not new;  in fact, it is the stationarity conditions in the Dubovitskii–Milyutin’s form with some refinements for our specific problem A. The novelty is only in the way of obtaining this result.

Remark 9.2**.**

If the functions $\varphi_{s}(u),\;s=1,\ldots,d(\varphi)$ are convex and the function $H(z^{0},x^{0},u)$ turns out to be concave in $u,$ then, as is known, stationarity condition (47) is equivalent to the maximality condition over the set $U=\{u\;|\;\varphi_{s}(u)\leq 0,$ $s=1,\ldots,d(\varphi)\}:$

[TABLE]

i.e., the necessary conditions for the extended weak minimality and for the strong minimality are equivalent. However, if the cost $J$ is not convex, then neither strong, nor even weak minimality can be guaranteed.

Remark 9.3**.**

Note that, in the proof of theorem 9.1, the variation of the reference process are made in two stages, not in one, as usual. First, we use not the whole class of possible variations, but only those for which $\bar{x}=\mbox{const}$ on the boundary interval $\Delta_{2}\,.$ In the second stage, we consider the stationarity conditions obtained for this reduced class, and substitute to them the “remaining” variations $\bar{x}\geq 0$ concentrated inside the boundary interval $\Delta_{2}$ and near its endpoints, which makes it possible to specify these conditions. This approach might be feasible not only for the given class of problems, but also for some other problems (see, e.g. Sec. 12–14 below).

10 On the Jumps of Measure – the Multiplier at the State Constraint

Of special interest is the question, in which cases the adjoint variable $\widetilde{\psi}_{x}(t)$ and the function $\mu(t)$ generating the measure do not have jumps at junction points?  Studies show (see, e.g. the book [19, §6] or papers [5, 20, 15, 21, 22, 23]) that in case of strong (or at least Pontryagin type [16, 17]) minimality, the adjoint variable and measure do not have jumps under condition (2). However, this result is not, in general, valid in the case of extended weak minimality (the reason is that one cannot rely upon the maximality of Pontryagin function w.r.t. $u,$ having in disposal only the stationarity of the extended Pontryagin function). Here, we specify a class of problems where the adjoint variable and measure have no jumps, and also present an example where the adjoint variable and measure corresponding to a stationary (but not optimal) trajectory do have nonzero jumps at junction points.

10.1 On the Absence of Atoms of Measure

Consider the case when the dynamics of the “free” state variable $z$ does not depend on $u:\;$ $\dot{z}=f(z,x).$ Applying (47) to the interval $\Delta_{2}\,,$ we get

[TABLE]

whence, in view of assumption $g^{\prime}_{u}(z^{0},x^{0},u^{0})\neq 0,$ obtain $\widetilde{\psi}_{x}\equiv 0$ on $\Delta_{2}\,.$ Thus, $\widetilde{\psi}_{x}(t_{1}-0)+\Delta\widetilde{\psi}_{x}(t_{1})=\widetilde{\psi}_{x}(t_{1}+0)=0,$ and so $\Delta\widetilde{\psi}_{x}(t_{1})=-\widetilde{\psi}_{x}(t_{1}-0).$

According to the energy conservation law (46), the jump of the so-called switching function (the $u$-dependent term of Pontryagin function) at the point $t_{1}$ is zero:

[TABLE]

where $g(t_{1}\pm 0):=g\left(z(t_{1}),x(t_{1}),u(t_{1}\pm 0)\right)\neq 0$ according to (2), and therefore $\Delta\widetilde{\psi}_{x}(t_{1})=0.$ One can similarly show that $\Delta\widetilde{\psi}_{x}(t_{2})=0$ either.

Thus, in the considered case, the measure has no atoms, and the adjoint variables are continuous. In the general case, the question of presence or absence of atoms is open. We leave it for further research.

10.2 An Example Where the Measure Has Atoms

Consider the following problem:

[TABLE]

where $z,\,x,\,u\in\mathbb{R},$ and a parameter $a>0$ is arbitrary. Let $\Delta_{1}=[0,1],$ $\Delta_{2}=[1,2],$ $\Delta_{3}=[2,3].$ Consider a trajectory generated by the control $u=(-1,0,1)$ on $\Delta_{1},\Delta_{2},\Delta_{3},$ for which $x=(1-t,\;0,\;t-2)$ on $\Delta_{1},\Delta_{2},\Delta_{3},$ respectively. The value of $z$ is defined up to an additive constant, which does not matter.

Let this trajectory satisfy the stationarity conditions of Theorem 9.1, i.e., let there exist Lipschitz continuous function $\psi_{z},$ Lipschitz continuous on $\Delta_{1},\Delta_{2},\Delta_{3}$ functions $\psi_{x}$ and $\mu$ with possible jumps at $t_{1}=1$ and $t_{2}=2,$ a constant $c,$ and a measurable bounded function $h,$ which generate the Pontryagin function $H=\psi_{z}f(u)+\psi_{x}u$ and the extended Pontryagin function

[TABLE]

such that the following condition hold:

1. (a)

adjoint equations

[TABLE] 2. (b)

transversality conditions

[TABLE] 3. (c)

complementary slackness conditions

[TABLE] 4. (d)

stationarity conditions w.r.t. control

[TABLE]

that imply the adjoint variable to be as follows

[TABLE] 5. (e)

and the energy conservation law

[TABLE]

From (50)–(51) it follows that $\psi_{z}\equiv 1.$ Set $h=(1,\,0,\,1)$ on $\Delta_{1},\Delta_{2},\Delta_{3}\,.$ The complementary slackness conditions are then obviously hold. Thus, according to (53), we get

[TABLE]

while the energy conservation law reads as follows:

[TABLE]

Conditions (56) are definitely satisfied if, e.g., $f$ is such that

[TABLE]

Then, the transversality conditions (51) hold too, and the jumps of $\psi_{x}$ at the points $1$ and $2$ are

[TABLE]

Now, it remains to find a smooth function $f$ satisfying conditions (57).

To this purpose one can use, e.g. the following polynomial:

[TABLE]

Thus, we get a stationary trajectory for which the adjoint variable $\psi_{x}$ has jumps $-a$ at the points $t=1,\,2.$ (Choosing a corresponding $f,$ one can make these jumps not equal.)

Note that here the Pontryagin function $H$ is not concave in $u$ (on the contrary, it is convex), so the stationarity conditions w.r.t. control $\overline{H}^{\prime}_{u}=0$ does not ensure the maximum of $H,$ i.e., the reference trajectory does not satisfy the maximum principle, and hence, it is just stationary but does not provide the strong minimality.

Thus, the stationarity conditions do not guarantee the absence of atoms, while, according to [5, 20, 19, 15, 21, 22, 23]), the maximum principle does. If a trajectory is not just stationary, but provides the strong (or at least Pontryagin type) minimality, then it satisfies the maximum principle, and therefore, the corresponding measure cannot have atoms.

11 An Example Where the Measure Has a Negative Density

Let us present an example showing that the condition of non-negativity of the measure density is essential, i.e., it does not follow from other stationarity conditions. Consider the following problem:

[TABLE]

Here $z=(z_{1},z_{2})\in\mathbb{R}^{2},$ the parameters $0<a<b<T$ are fixed, while the parameters $\widehat{z}_{1},\,\widehat{z}_{2},\;\widehat{x}_{0},\,\widehat{x}_{T}$ are also fixed and will be defined below. The function $f=(f_{1},f_{2})=((z_{2}-a)(z_{2}-b)x,\,1)),$ thus $f^{\prime}_{x}=((z_{2}-a)(z_{2}-b),\,0)).$ The endpoints of the trajectory are free.

Consider a trajectory with $u^{0}=(-1,0,1)$ on the intervals $[0,a],$ $[a,b],$ $[b,T],$ respectively, $z_{1}^{0}(0)=0,$ $z_{2}^{0}(t)\equiv t,$ and $x^{0}(t)=0$ on $[a,b].$ Thus, $x^{0}(t)=a-t>0$ for $t<a$ and $x^{0}(t)=t-b$ for $t>b.$ Check, whether stationarity conditions (43)–(47) hold.

The extended Pontryagin function is

[TABLE]

where, in view of the complementary slackness conditions, $\dot{\mu}=0$ outside of $[a,b],$ and $h=0$ on $[a,b].$

The adjoint equations and transversality conditions are as follows:

[TABLE]

By the first equation, $\psi_{z_{1}}\equiv-1,$ hence, if we set $\widehat{z}_{1}=1/2,$ the transversality condition for $\psi_{z_{1}}$ is satisfied.

From (59), we get equations for $\psi_{z_{2}}$:

[TABLE]

Solving the initial value problems on $[0,a]$ and $[b,T],$ we get

[TABLE]

and, since $\psi_{z_{2}}$ is continuous everywhere and constant on $[a,b],$ it should satisfy the equality $\psi_{z_{2}}(a-0)=\psi_{z_{2}}(b+0),$ i.e.,

[TABLE]

Obviously, there exists such $\widehat{z}_{2}$ that it holds. Fix this $\widehat{z}_{2}$.

Similarly, for $\widetilde{\psi}_{x}$ we get from (59):

[TABLE]

Solving the initial value problems on $[0,a]$ and $[b,T]$ with $\dot{\mu}=0,$ we get

[TABLE]

The condition $\overline{H}_{u}\equiv 0,$ i.e., $\widetilde{\psi}_{x}\equiv 2hu^{0},$ implies $\widetilde{\psi}_{x}\equiv 0$ on $[a,b].$ According to (45), $\Delta\widetilde{\psi}_{x}(a)\leq 0,$ hence $\widetilde{\psi}_{x}(a-0)\geq 0.$ If $\widetilde{\psi}_{x}(a-0)>0,$ then $h<0$ in a left neighborhood of $a,$ a contradiction with $h\geq 0.$ Therefore, $\widetilde{\psi}_{x}(a-0)=0.$ Similarly, we get $\widetilde{\psi}_{x}(b+0)=0,$ i.e., $\widetilde{\psi}_{x}$ has no jumps at $t=a$ and $t=b.$

The fulfillment of the obtained equalities is equivalent to the following linear relations on the parameters $\widehat{x}_{0},\,\widehat{x}_{T}$:

[TABLE]

Obviously, such $\widehat{x}_{0},\,\widehat{x}_{T}$ do exist. Fix these values.

Finally, from (63) it follows that $\dot{\widetilde{\psi}}_{x}>0$ on $(0,a)$ and $(b,T),$ so $\widetilde{\psi}_{x}<0$ on $[0,a)$ and $\widetilde{\psi}_{x}>0$ on $(b,T],$ and then the condition $\widetilde{\psi}_{x}=2hu^{0}$ implies that $h(t)>0$ on these intervals. Thus, for the chosen parameters of problem and for the examined trajectory, there exists a unique collection of multipliers satisfying all the conditions of Theorem 9.1 except (41). Here, condition (63) implies that $\dot{\mu}=(t-a)(t-b)<0$ on $(a,b),$ which contradicts the condition (41). Thus, the last condition does not follow from the others, and the examined trajectory does not provide the extended weak minimality.

12 Generalization of the Obtained Result

An important feature of problem (1) is that the state constraint has the form $x\geq 0,$ i.e., it is imposed only on one state coordinate. Let us show how it is possible to use the above result to formulate stationarity conditions in a more general

[TABLE]

Here $y\in\mathbb{R}^{n+1},$ $u\in\mathbb{R}^{m},$ the state variable $y(\cdot)$ is absolutely continuous, and the control $u(\cdot)$ is measurable bounded functions. We assume that the data functions $f,\,\varphi,$ and $\Phi$ are defined and twice continuously differentiable on an open subset $\mathcal{Q}\subset\mathbb{R}^{n+1+m}.$

As before, we suppose that the reference process $w^{0}=(y^{0},u^{0})$ is such that the trajectory $y^{0}(t)$ touches the state boundary only on a segment $[t^{0}_{1},t^{0}_{2}],$ where $0<t^{0}_{1}<t^{0}_{2}<T.$ In other words, the interval $\Delta:=[0,T]$ is divided into three parts $\Delta_{1}:=[0,t^{0}_{1}],$ $\Delta_{2}:=[t^{0}_{1},t^{0}_{2}],$ and $\Delta_{3}:=[t^{0}_{2},T],$ such that $\Phi(y^{0}(t))>0$ on $[0,t^{0}_{1}),$ $\Phi(y^{0}(t))=0$ on $\Delta_{2},$ and $\Phi(y^{0}(t))>0$ on $(t^{0}_{2},T].$ The control $u^{0}(t)$ is continuous on $\Delta_{1}\,,\Delta_{3}\,,$ Lipschitz continuous on $\Delta_{2},$ and, moreover, $\varphi_{s}(u^{0}(t))<0$ on $\Delta_{2}$ for all $s,$ and the landing to the state boundary and the leaving it occurs with nonzero time derivatives:

[TABLE]

As before, we assume that the gradients $\varphi^{\prime}_{i}(u^{0}(t)),\;\,i\in I(u^{0}(t))$ are positive independent for all $t\in\Delta_{1}\cup\Delta_{3}\,,$ and $\Phi^{\prime}(y^{0}(t))f_{u}(y^{0}(t),\,u^{0}(t))\neq 0$ on $\Delta_{2}$.

13 Reduction of Problem C to Problem A

We accept the following technical

Assumption C. There exist an open subset $\,\Omega\subset\mathbb{R}^{n+1}\,$ containing  the curve $y^{0}(t),\;\,t\in[0,T],$ and twice continuously differentiable functions $P_{i}:\Omega\to\mathbb{R},\;\,i=1,\ldots,n,$ such that the gradients $P_{1}^{\prime}(y),\ldots,P_{n}^{\prime}(y),\,\Phi^{\prime}(y)$ are linearly independent at any point $y\in\Omega,$ and, moreover, the mapping $F:\Omega\to\mathbb{R}^{n+1}$ defined by

[TABLE]

is an injection. In other words, $F$ realizes a nondegenerate change of variables in $\Omega$:

[TABLE]

Herewith, $det\,F^{\prime}(y^{0}(t))\neq 0,$ the set $Q=F(\Omega)$ is also open, and there exists a inverse mapping $G:Q\to\Omega,$ $(z,x)\mapsto y,$ so that

[TABLE]

In what follows, we will always assume that $y,\,z,\,x$ satisfy the following relations

[TABLE]

Note that differentiation of (69) yields the equality

[TABLE]

where the right hand part is the identity matrix of dimension $n+1$.

Remark 13.1**.**

It is sufficient to assume that $\Omega$ contains not the entire curve $y^{0}(t),$ $t\in[0,T],$ but only part of it for $t\in\Delta_{2}.$ Then, by extending the definition of the function $P$ out of $\Omega,$ one can reduce the situation to the case of $\Omega$ containing the entire curve $y^{0}(t).$ Here we do not dwell on the corresponding technical details. Note only that Assumption C is really satisfied in all reasonable, especially applied, problems with state constraints.

Obviously, the dynamics of state variables $z,\,x$ obeys the system

[TABLE]

therefore, problem (65) in these new variables transforms to the following problem of type (1) on the same time interval $[0,T]$:

[TABLE]

To each process $w=(y(t),u(t))$ of problem C  one can associate a process $\gamma=(z(t),x(t),u(t))$ of problem D, and vice versa. Obviously, the process $w^{0}$ provides the extended weak minimality in problem C if and only if the corresponding process $\gamma^{0}$ provides the extended weak minimality in problem D.

Therefore, we can use the fact that the process $\gamma^{0}$ satisfies the stationarity conditions given in Theorem 9.1.

14 Stationarity Conditions for Problem C

In further transformations, we have to differentiate vector-valued and matrix-valued functions w.r.t a vector argument. To avoid cumbersome formulas in the coordinate form, let us accept the following notation. If $T(z)$ is any tensor of a given rank (in particular, a vector or a matrix), every element $\theta(z)$ of which is a smooth function of $z\in\mathbb{R}^{n},$ then its directional derivative along a vector $\bar{z}\in\mathbb{R}^{n}$ will be denoted as $T^{\prime}(z)\,\bar{z}.$ The last one is still a tensor of the same rank and dimension, whose elements $\theta^{\prime}(z)\,\bar{z}=\sum_{i=1}^{n}\theta^{\prime}_{z_{i}}(z)\,\bar{z}_{i}$ are the scalar directional derivatives of the corresponding elements $\theta(z)$ along the vector $\bar{z}$.

According to Theorem 9.1, if the process $\gamma^{0}=(z^{0}(t),x^{0}(t),u^{0}(t))$ provides the extended weak minimality in problem D, then there exist a Lipschitz continuous adjoint variable $\psi_{z}(t)$ $(n-$dimensional row vector) on $[0,T],$ a constant $c,$ scalar functions $\mu(t)$ and $\psi_{x}(t),$ Lipschitz continuous on each interval $\Delta_{i},$ $i=1,2,3,$ such that $d\mu(t)\geq 0,$ and a measurable bounded function $h(t)\geq 0,$ which generate the Pontryagin function

[TABLE]

and the extended Pontryagin function

[TABLE]

such that the following conditions hold:

complementary slackness

[TABLE]

adjoint equations

[TABLE]

[TABLE]

(these equalities hold for any “test” constant vectors $\overline{z}\in\mathbb{R}^{n}$ and $\overline{x}\in\mathbb{R}^{1}),$

transversality conditions

[TABLE]

jump conditions for the adjoint variable $\psi_{x}$

[TABLE]

the energy conservation law

[TABLE]

and stationatity condition w.r.t. control

[TABLE]

Now, rewrite the obtained conditions in terms of problem C. First, denote

[TABLE]

This is a row vector of dimension $n+1.$ Then, since $G(z^{0},x^{0})=y^{0},$ condition (79) takes the form

[TABLE]

Further, multiplying $\psi_{y}$ by a test (constant) vector $\overline{y}\in\mathbb{R}^{n+1},$ we get a scalar function

[TABLE]

(for short, we drop the arguments of $G$ and $f$), which time derivative is

[TABLE]

where $(...)^{\bullet}$ denotes the time derivarive of the function in brackets.

Let us write the first two terms of this expression in view of equations (74) and (75) for $\bar{z}=P^{\prime}(G)\bar{y},$ $\;\bar{x}=\Phi^{\prime}(G)\bar{y}$:

[TABLE]

[TABLE]

The other two terms of (82) in view of identities $\dot{G}=\dot{y}=f$ are equal to

[TABLE]

Summing up the right parts of equalities (83)–(85), we get

[TABLE]

Note that the matrix $\psi_{z}P^{\prime\prime}(G)+\psi_{x}\Phi^{\prime\prime}(G)$ is the second derivative of the scalar function $\psi_{z}P(G)+\psi_{x}\Phi(G),$ hence it is symmetric. Therefore, the first and the last terms in the right hand part of obtained expression (which differ only in the positions of multipliers $\overline{y}$ and $f$) cancel each other, and in view of relation $\bar{x}=\Phi^{\prime}(G)\bar{y},$ equation (86) takes the form

[TABLE]

whence, since the test vector $\bar{y}\in\mathbb{R}^{n+1}$ is arbitrary, we get

[TABLE]

If we introduce the Pontryagin function $H=\psi_{y}f(y,u)$ and the extended Pontryagin function $\overline{H}=\psi_{y}f(y,u)+\dot{\mu}\Phi(y)-h\varphi(u)$ for problem C, then equalities (81) and (87) transform to $\overline{H}_{u}=0$ and $-\dot{\psi}_{y}=\overline{H}_{y}\,$ respectively.

According to (77), the function $\psi_{y}$ has jumps at the points $t_{1}^{0},\,t_{2}^{0}$:

[TABLE]

The transversality conditions for $\psi_{y}$ take the form

[TABLE]

Finally, the complementary slackness conditions and the energy conservation law are rewritten automatically in terms of problem C.

Summarizing our findings, we come to the following

Theorem 14.1**.**

Let $w^{0}=(y^{0}(t),u^{0}(t)$ be an admissible process such that $\Phi(y^{0}(t))=0$ on $\Delta^{0}_{2}:=\left[t^{0}_{1},t^{0}_{2}\right],$ $\Phi(y^{0}(t))>0$ on $[0,T]\setminus\Delta^{0}_{2},$ ${\;\varphi_{i}(u^{0}(t))<0}$ on $\Delta_{2}\,,$ assumption (66) holds, and let this process provide the extended weak minimality in problem C.  Then there exist a constant $c,$ functions $\psi_{y}(t),$ $\mu(t)$ Lipschitz continuous on every interval $\Delta_{i},\;i=1,2,3,$ and a measurable bounded function $h(t),$ which generate the Pontryagin function

[TABLE]

and the extended Pontryagin function

[TABLE]

such that the following conditions hold:

1. (a)

non-negativity conditions

[TABLE] 2. (b)

complementary slackness

[TABLE] 3. (c)

adjoint equation

[TABLE] 4. (d)

transversality conditions

[TABLE] 5. (e)

jumps conditions for the adjoint variable

[TABLE] 6. (f)

energy conservation law

[TABLE] 7. (g)

and stationarity condition w.r.t. control

[TABLE]

Remark 14.1**.**

The performed transformation $y\mapsto(z,x)$ is a particular case of the general one-to-one change of variables $w=F(y),\;y=G(w),$ under which problem C transforms to the following

[TABLE]

Clearly, the extended weak minimality at a process $(y^{0},u^{0})$ in problem C corresponds to that at the process $(w^{0}=F(y^{0}),\,u^{0})$ in problem E. The multipliers $\alpha_{0},\,h,\,\mu$ in both problems are the same, the extended Pontryagin functions for problems C and E are, respectively,

[TABLE]

while the adjoint variables are connected by the following equality:

[TABLE]

The proof of this assertion is left to the reader as an excercise.

In the case of problem D, we have $w=(z,x)$ and $F=(P,\Phi),$ hence

[TABLE]

i.e., we get exactly formula (80).

Remark 14.2**.**

For simplicity, we considered problem A  with free endpoints of the trajectory. If they are restricted by terminal constrains

[TABLE]

then, to obtain stationarity conditions, one should replace the cost $J$ by the endpoint Lagrange function $l=\alpha_{0}J+\sum_{k}\alpha_{k}\xi_{k}+\sum_{j}\beta_{j}\eta_{j}$ (with corresponding multipliers) and then apply Theorem 9.1.  The same concerns problem C.

Remark 14.3**.**

We suppose that the state constraint in problem (1) or (65) is of first order, and the reference trajectory lands on the state boundary with a nonzero first time derivative, i.e., satisfies conditions (2) or (66), respectively. Perhaps, the same approach would also work in the case of higher order state constraints, if the reference trajectory lands on the state boundary with a nonzero time derivative of the corresponding order. Obviously, the technique would be then more complicated.

15 Conclusions

We consider a specific class of optimal control problems with a single state constraint of order 1  and a specific trajectory in it. Basing on the approach by R.V. Gamkrelidze, consisting in differentiating the state constraint along the boundary subarc and reducing the original problem to a problem with mixed control-state constraints, we obtain the full system of stationarity conditions in the form of A.Ya. Dubovitskii and A.A. Milyutin, including the sign definiteness of the measure, a multiplier at the state constraint. To obtain these conditions, we propose an approach of two-stage varying. At the first stage, we consider only those variations, which preserve a constant value of the state constraint along the boundary interval, and obtain preliminary, incomplete optimality conditions. At the second stage, we take into account the remaining variations, concentrated on the boundary interval, and obtain the sign definiteness of the measure, thus specifying the stationarity conditions. Two illustrative examples are given, one showing that the condition of non-negativity of the measure density is essential and another with nonzero atoms of the measure at the junction points.

Acknowledgements 15.1**.**

This research was partially supported by the Russian Foundation for Basic Research under grant No. 16-01-00585.  The authors thank Nikolai Osmolovskii for useful discussions and the anonymous referees for valuable remarks.