Optimal control on distributions

TL;DR

This paper explores optimal control problems on nonholonomic manifolds, analyzing various formulations and proving that such systems can be controlled with bang-bang controls in single or multiple time settings.

Contribution

It introduces a comprehensive analysis of optimal control on nonholonomic manifolds, including infinitesimal deformations, adjointness, and control strategies for complex functionals.

Findings

Nonholonomic systems can be controlled by bang-bang controls in single or bi-temporal settings.

Analysis of infinitesimal deformations and adjointness in control problems.

Extension to multitime optimal control problems with various functional types.

Abstract

This paper studies (single-time and multitime) optimal control problems on a nonholonomic manifold (described either by the kernel of a Gibbs-Pfaff form or by the span of appropriate vector fields). For both descriptions we analyse: infinitesimal deformations and adjointness, single-time optimal control problems, multitime optimal control problem of maximizing a multiple integral functional, multitime optimal control problem of maximizing a curvilinear integral functional, Curvilinear functionals depending on curves, optimization of mechanical work on Riemannian manifolds. Also we prove that a nonholonomic system can be always controlled by uni-temporal or bi-temporal bang-bang controls.

Equations

$$\frac{d}{dt}x(t)=f(x(t),x(t-\tau )).$$

$$a_i(x(t),x(t-\tau ))d{x}^i(t)=0.$$

$$a_i(x)d{x}^i=0.$$

$$A_i(x(t;ϵ);ϵ)d{x}^i(t;ϵ)=0,A_i(x;0)=a_i(x),x(t;0)=x(t).$$

$$\left(\frac{\partial a_i}{\partial x^j}(x)y^j(t)+b_i(x)\right)d{x}^i+a_i(x)d{y}^i=0,b_i(x)=\frac{\partial A_i}{\partial ϵ}(x;0)$$

$$d(p(t)a_j(x(t)))=\frac{\partial (p(t)a_i(x)d{x}^i)}{\partial x^j},$$

$$A_i(x(t;ϵ);ϵ)d{x}^i(t;ϵ)=0,A_i(x,0)=a_i(x),x(t;0)=x(t).$$

$$\left(\frac{\partial a_i}{\partial x^j}(x)y_A^j(t)+b_{iA}(x)\right)d{x}^i+a_i(x)d{y}_A^i=0,b_{iA}(x)=\frac{\partial A_i}{\partial {ϵ}^A}(x;0)$$

$$d(p^A(t)a_j(x))=\frac{\partial (p^A(t)a_i(x)d{x}^i)}{\partial x^j},$$

$$A_i(x(t;ϵ),ϵ)d{x}^i(t;ϵ)=B(x(t;ϵ),ϵ)dϵ,$$

$$A_i(x,0)=a_i(x),B(x,ϵ)=A_i(x,ϵ)\frac{\partial x^i}{\partial ϵ},B(x,0)=0,x(t;0)=x(t).$$

$$\left(\frac{\partial A_i}{\partial x^j}\frac{\partial x^j}{\partial ϵ}+\frac{\partial A_i}{\partial ϵ}\right)d{x}^i+A_{i}d\frac{\partial x^i}{\partial ϵ}=\left(\frac{\partial B}{\partial x^j}\frac{\partial x^j}{\partial ϵ}+\frac{\partial B}{\partial ϵ}\right)dϵ.$$

$$\left(\frac{\partial A_i}{\partial x^j}{X}^j+\frac{\partial A_i}{\partial ϵ}\right)d{x}^i+A_{i}d{X}^i=\left(\frac{\partial B}{\partial x^j}{X}^j+\frac{\partial B}{\partial ϵ}\right)dϵ$$

$$a_i(x,u)d{x}^i=0,x=(x^i)\in R^n,u=(u^a)\in R^k$$

$$I(u(\cdot ))={\int }_{t_0}^{t_1}L(t,x(t),u(t))dt+g(x(t_1))$$

$$a_i(x(t),u(t))d{x}^i(t)=0,a.e.t\in I=[t_0,t_1],x(t_0)=x_0.$$

$$\frac{\partial a_i}{\partial x^j}(x,u)y^{j}d{x}^i+a_i(x,u)d{y}^i=0.$$

$$d(p(t)a_j(x(t),u(t)))=\frac{\partial (p(t)a_i(x,u)d{x}^i)}{\partial x^j},$$

$$\mathcal{L}=L(t,x(t),u(t))dt,$$

$$\mathcal{H}=\mathcal{L}(t,x(t),u(t))+p(t)a_i(x(t),u(t))d{x}^i(t).$$

$$\frac{\partial \mathcal{H}}{\partial p}=a_i(x(t),u(t))d{x}^i(t)=0,$$

$$d(p(t)a_j(x(t),u(t)))=\frac{\partial \mathcal{H}}{\partial x^j}$$

$${\mathcal{H}}_{u^a}\left(t,x(t),u(t),p(t)\right)=0,a=\overline{1,k}.$$

$$J(u(\cdot ))={\int }_{\tilde{\Gamma }}\mathcal{H}(t,x(t),u(t),p(T))+g(x(t_1)),$$

$$a_i(x(t;ϵ),u(t;ϵ))d{x}^i(t;ϵ)=0,x(t;0)=x(t),x(t_0,0)=x_0.$$

$$J(ϵ)={\int }_{\tilde{\Gamma }(ϵ)}\mathcal{H}(t,x(t,ϵ),u(t,ϵ),p(t))+g(x(t_1,ϵ))$$

$$={\int }_{\tilde{\Gamma }(ϵ)}\mathcal{L}(t,x(t,ϵ),u(t,ϵ))+p(t)a_i(x(t,ϵ),u(t,ϵ))d{x}^i(t,ϵ)+g(x(t_1,ϵ)).$$

$$J^{\prime }(ϵ)={\int }_{\tilde{\Gamma }(ϵ)}{\mathcal{L}}_{x^j}(t,x(t,ϵ),u(t,ϵ))x_{ϵ}^j(t,ϵ)+\frac{\partial g}{\partial x^j}(x(t_1,ϵ))x_{ϵ}^j(t_1,ϵ)$$

$$+{\int }_{\tilde{\Gamma }(ϵ)}p(t)\frac{\partial a_i}{\partial x^j}(x(t,ϵ),u(t,ϵ))x_{ϵ}^j(t,ϵ)d{x}^i(t,ϵ)$$

$$+{\int }_{\tilde{\Gamma }(ϵ)}p(t)a_i(x(t,ϵ),u(t,ϵ))d{x}_{ϵ}^i(t,ϵ)$$

Full text

Optimal control on distributions

Constantin Udrişte

This paper studies (single-time and multitime) optimal control problems on a nonholonomic manifold (described either by the kernel of a Gibbs-Pfaff form or by the span of appropriate vector fields). For both descriptions we analyse: infinitesimal deformations and adjointness, single-time optimal control problems, multitime optimal control problem of maximizing a multiple integral functional, multitime optimal control problem of maximizing a curvilinear integral functional, Curvilinear functionals depending on curves, optimization of mechanical work on Riemannian manifolds. Also we prove that a nonholonomic system can be always controlled by uni-temporal or bi-temporal bang-bang controls.

Mathematics Subject Classification 2010: 49J15, 49J20, 93C15, 93C20.

Keywords: nonholonomic manifold, single-time optimal control, multitime optimal control, bang-bang controls.

1 Optimal control on a distribution

described by a Pfaff equation

A generalized distribution $\Delta:x\to\Delta_{x}$, or Stefan-Sussmann distribution, is similar to a distribution, but the subspaces are not required to be all of the same dimension. The definition requires that the subspaces $\Delta_{x}$ are locally spanned by a set of vector fields, but these will no longer be everywhere linearly independent. It is not hard to see that the dimension of the distribution $\Delta$ is lower semicontinous, so that at special points the dimension is lower than at nearby points. One class of examples is provided by a non-free action of a Lie group on a manifold, the vector fields in question being the infinitesimal generators of the group action (a free action gives rise to a genuine distribution). Another examples arise in dynamical systems, where the set of vector fields in the definition is the set of vector fields that commute with a given one. There are also examples and applications in Control theory, where the generalized distribution represents infinitesimal constraints of the system.

see ControlJakubczyk, pag 146

Lemma If the variational system (treated as linear system without constraints on the control) is controllable, then the original system is strongly accessible.

Nonholonomic path planning represents a fusion of some of the newest ideas in control theory, classical mechanics, and differential geometry with some of the most challenging practical problems in robot motion planning. Furthermore, the class of systems to which the theory is relevant is broad: mobile robots, space-based robots, multifingered hands, and even such systems as a one-legged hopping robot. The techniques presented here indicate one possible method for generating efficient and computable trajectories for some of these nonholonomic systcms in the absence of obstacles.

The delay differential equations (DDEs) are a type of differential equation in which the derivative of the unknown function at a certain time is given in terms of the values of the function at previous times. For example

[TABLE]

A delay Pfaff equation means

[TABLE]

1.1 Infinitesimal deformations and

adjointness on distributions

Let $D$ be a nonholonomic distribution on $R^{n}$ described by a Pfaff equation

[TABLE]

Let $x(t),\,\,t\in I=[t_{0},t_{1}]\subset R$, be an integral curve of the distribution $D$. Let $x(t;\epsilon),\,\epsilon\in[0,\delta)$ be a differentiable variation of $x(t)$, i.e.,

[TABLE]

The variation $(t,\epsilon)\to x(t;\epsilon),\,t\in I,\,\epsilon\in[0,\delta)$ is a surface. It is an integral surface only if the distribution $D$ admits integral surfaces. Taking the partial derivative with respect to $\epsilon$ and denoting $y^{i}(t)=\displaystyle\frac{\partial x^{i}}{\partial\epsilon}(t;0)$, we find the single-time (Pfaff) infinitesimal deformation equation

[TABLE]

around a solution $x(t)$ of the Pfaff equation (1). The single-time adjoint Pfaff system is

[TABLE]

whose solution $p(t)$ is called the costate function. The foregoing Pfaff equations (2) and (3) are adjoint (dual) in the following sense: if $y$ is a solution of the infinitesimal deformation (Pfaff) equation (2), then the function $p(t)a_{i}(x)y^{i}(t)$ verifies the Pfaff equation $d(p(t)\,a_{i}(x)y^{i}(t))+p(t)\,b_{i}(x)dx^{i}=0$.

Let $x(t),\,\,t\in\Omega_{t_{0}t_{1}}\subset R^{m}_{+}$ be a maximal, $m$-dimensional, $m\leq n-2$, integral submanifold of the distribution $D$. We fix $k=1,...,n-m-1$. Let $\epsilon=(\epsilon^{A})\in[0,\delta)^{k},\,A=1,...,k,$ and let $x(t;\epsilon)$ be a differentiable variation of $x(t)$, i.e.,

[TABLE]

The variation $(t,\epsilon)\to x(t;\epsilon),\,t\in\Omega_{t_{0}t_{1}},\,\epsilon\in[0,\delta)^{k}$ is an $(m+k)$-dimensional manifold, but not an integral submanifold.

Taking the partial derivative with respect to $\epsilon^{A}$ and denoting $y^{i}_{A}(t)=\frac{\partial x^{i}}{\partial\epsilon^{A}}(t;0)$, we find the multitime infinitesimal deformation (Pfaff) system

[TABLE]

around a solution $x(t)$ of the Pfaff equation (1). The multitime adjoint Pfaff system is

[TABLE]

whose solution $p(t)=(p^{A}(t))$ is called the costate vector. The foregoing Pfaff equations (4) and (5) are adjoint (dual) in the following sense: if $y^{i}_{A}$ is a solution of the infinitesimal deformation (Pfaff) system (4), then the function $a_{i}(x)p^{A}(t)\,y^{i}_{A}(t)$ verify the Pfaff equation $d(a_{i}(x)p^{A}(t)\,y^{i}_{A}(t))+p^{A}(t)\,b_{iA}(x)dx^{i}=0$.

1.2 Evolution of a distribution

Let $D$ be a nonholonomic distribution on $R^{n}$ described by a Pfaff equation (1). Let $x(t),\,\,t\in I=[t_{0},t_{1}]\subset R$, be an integral curve of the distribution $D$. Let $x(t;\epsilon),\,\epsilon\in[0,\delta)$ be a differentiable variation of $x(t)$. Suppose that $(x(t;\epsilon);\epsilon)$ is an integral surface of a Pfaff equation in $R^{n+1}$, i.e.,

[TABLE]

[TABLE]

Taking the partial derivative with respect to $\epsilon$, we find

[TABLE]

If we accept an evolution after the direction of the vector field $X^{j}$, i.e., $\displaystyle\frac{\partial x^{j}}{\partial\epsilon}=\alpha\,X^{j}$, then we find the PDE system

[TABLE]

with unknowns $A_{i}$, fixed by initial conditions. For $\epsilon=0$, we rediscover the system in variations, with the condition $a_{i}(x)y^{i}(t)=0,\,y^{i}(t)=\frac{\partial x^{i}}{\partial\epsilon}(t,0)$.

1.3 Single-time optimal control problems on

a distribution

Let $D$ be a distribution on $R^{n}$ described by a controlled Pfaff equation

[TABLE]

and let $x(t),\,\,t\in I=[t_{0},t_{1}],$ be an integral curve of the distribution $D$.

A single-time optimal control problem consists of maximizing the functional

[TABLE]

subject to

[TABLE]

It is supposed that $L:I\times A\times U\to R$ is a $C^{2}$ function, $a_{i}:A\times U\to R,\,i=1,...,n$ are $C^{2}$ functions and $g(x(t_{1}))$ is a $C^{1}$ function. Ingredients: $A$ is a bounded and closed subset of $R^{n}$ which contains each trajectory $x(t),\,t\in I$ of controlled system, and $x_{0}$ and $x_{1}$ are the initial and final states of the trajectory $x(t)$. The values of the control functions belong to a set $U\subset R^{k}$, bounded and closed.

Let us find the first order necessary conditions for an optimal pair $(x,u)$. We fix the control $u(t)$ and we variate the state $x(t)$ into $x(t,\epsilon)$. We obtain the single-time infinitesimal deformation (Pfaff) equation

[TABLE]

of the nonholonomic constraint $a_{i}(x(t),u(t))dx^{i}(t)=0$. It follows the single-time adjoint Pfaff equation

[TABLE]

whose solution $p(t)$ is called the costate function. Here, the symbol $d$ in the left hand member of the adjoint equation means the differentiation with respect to $p$ and $x$.

Using the Lagrangian $1$-form

[TABLE]

we build the Hamiltonian $1$-form

[TABLE]

Theorem (Single-time maximum principle) Suppose that the problem of maximizing the functional (6) constrained by (7) has an interior optimal solution $\hat{u}(t)$, which determines the optimal evolution $x(t)$. Then there exists a costate function $(p(t))$ such that

[TABLE]

the function $(p(t))$ is the unique solution of the following Pfaff system (adjoint system)

[TABLE]

and satisfies the critical point conditions

[TABLE]

Proof We use the Hamiltonian $1$-form ${\cal H}$. The solutions of the foregoing problem are among the solutions of the free maximization problem of the curvilinear integral functional

[TABLE]

where $\tilde{\Gamma}=([t_{0},t_{1}],x([t_{0},t_{1}]))=\{(t,x(t))|t\in[t_{0},t_{1}]\}\subset R_{+}\times R^{n}$.

Suppose that there exists a continuous control $\hat{u}(t),\,t\in I=[t_{0},t_{1}]$, with $\hat{u}(t)\in\hbox{Int}\,{U}$, and an integral curve $x(t)$ which are optimal in the previous problem. Now consider a control variation $u(t,\epsilon)=\hat{u}(t)+\epsilon h(t),\,t\in I=[t_{0},t_{1}]$, where $h$ is an arbitrary continuous vector function, and a state variation $x(t,\epsilon),\,t\in I=[t_{0},t_{1}]$, related by

[TABLE]

Since $\hat{u}(t)\in\hbox{Int}\,{\cal U}$ and any continuous function over a compact set $I$ is bounded, there exists $\epsilon_{h}>0$ such that $u(t,\epsilon)=\hat{u}(t)+\epsilon h(t)\in\hbox{Int}\,{U},\,\,\forall|\epsilon|<\epsilon_{h}$. This $\epsilon$ is used in our variational arguments.

For $|\epsilon|<\epsilon_{h}$, we define the function

[TABLE]

[TABLE]

Differentiating with respect to $\epsilon$, it follows

[TABLE]

[TABLE]

[TABLE]

[TABLE]

[TABLE]

Evaluating at $\epsilon=0$, we find $\tilde{\Gamma}(0)=\tilde{\Gamma}$ and

[TABLE]

[TABLE]

[TABLE]

where $x(t)$ is the curve of the state variable corresponding to the optimal control $\hat{u}(t)$. The integral from the middle can be written

[TABLE]

where the symbol $d$ in the last integral means the differentiation with respect to $p$ and $x$. We find $J^{\prime}(0)$ as

[TABLE]

[TABLE]

[TABLE]

Since $x(t_{0})=x_{0}$, we have $x^{j}_{\epsilon}(t_{0},0)=0$. We select the costate function $p(t)$ as solution of the adjoint Pfaff equation

[TABLE]

with the terminal condition $p(t_{1})a_{j}(x(t_{1}),\hat{u}(t_{1}))+\frac{\partial g}{\partial x^{j}}(x(t_{1}))=0$. On the other hand, we need $J^{\prime}(0)=0$ for all $h(t)=(h^{a}(t))$. Since the variation $h$ is arbitrary, we get the following (critical point condition)

[TABLE]

The foregoing equations (9) and (10) can be written

[TABLE]

[TABLE]

Example Let us solve the problem

[TABLE]

subject to (controlled Martinet distribution)

[TABLE]

Denote $\omega=\frac{1}{2}(y^{2}+u)dx-dz$. Since

[TABLE]

the distribution admits only integral curves with the parameter $t$. The Pfaff equation

[TABLE]

is equivalent to a differential equation

[TABLE]

or to the ODE system

[TABLE]

First variant Using the Hamiltonian

[TABLE]

we find the adjoint ODEs

[TABLE]

and the critical point condition

[TABLE]

We find the control $u=\frac{1}{2}\,p_{3}\varphi$ and $p_{1}(t)=c_{1}$. We need to find solutions for the system

[TABLE]

Second variant The Hamiltonian $1$-form is

[TABLE]

The critical point condition

[TABLE]

gives the control $u=\frac{1}{2}\,p\dot{x}$.

Since $a_{1}=\frac{1}{2}(y^{2}+u)$, $a_{2}=0$, $a_{3}=-1$, we find the adjoint Pfaff equations

[TABLE]

We need to solve the system

[TABLE]

[TABLE]

On the other hand, the second variant ofers two explicit extremals: (1) $y=0,pu=c_{1},dp=zdt,udt=\frac{1}{2}pdx,dz=udx$, which does not satisfy the general initial conditions; (2) $x=c_{1},u=0,z=c_{3},p=c_{3}t+c_{4},y=\pm\sqrt{\frac{c}{c_{3}t+c_{4}}}$, depending upon four arbitrary constants, which determine from initial conditions and terminal condition.

The first variant can be identified with the second variant via $p_{1}=pa_{1};p_{2}=pa_{2},p_{3}=-pa_{3}$.

Third variant (Ionel Tevy) We introduce two auxiliary controls $u_{1},u_{2}$, changing the Pfaff equation into the controlled ODE system

[TABLE]

Then

[TABLE]

We find the adjoint ODEs and the critical point conditions

[TABLE]

[TABLE]

It follows two extremals:

[TABLE]

[TABLE]

[TABLE]

[TABLE]

[TABLE]

[TABLE]

O varianta a rezolvarii Tevy se gaseste in Michele Pavon, Optimal control of nonholonomic systems, Proceedings of the 17th International Symposium on Mathematical Theory of Networks and Systems, Kyoto, July 24-28, 2006; vezi NonhOptCon.pdf; citeaza lucrarile mele;

1.4 Multitime optimal control problems on

a distribution

Let $D$ be a distribution on $R^{n}$ described by a controlled Pfaff equation

[TABLE]

and let $x(t),\,\,t\in\Omega_{t_{0}t_{1}}\subset R^{m}$ be an $m$-dimensional ($m<n$, m maximal) integral submanifold of the distribution $D$.

Let us start with a

1.4.1 Multitime optimal control problem of

maximizing a multiple integral functional

Find

[TABLE]

subject to

[TABLE]

It is supposed that $L:\Omega_{t_{0}t_{1}}\times A\times U\to R$ is a $C^{2}$ function and $a_{i}:A\times U\to R,\,i=1,...,n$ are $C^{2}$ functions. Ingredients: $\omega=dt^{1}\cdots dt^{m}$ is the volume element, $A$ is a bounded and closed subset of $R^{n}$, containing the images of the $m$-sheets $x(t),\,\ t\in\Omega_{t_{0}t_{1}}$ of controlled system, and $x_{0}$ and $x_{1}$ are the initial and final states of an $m$-sheet $x(t)$. The set in which the control functions takes their values is called as $U$, which is a bounded and closed subset of $R^{k}$.

Let us find the first order necessary conditions for an optimal pair $(x,u)$. We fix the control $u(t)$ and variate the state $x(t)$ into $x(t,\epsilon),\,\epsilon=(\epsilon^{1},...,\epsilon^{m})$. We find the multitime infinitesimal deformation (Pfaff) system

[TABLE]

of the nonholonomic constraint $a_{i}(x(t),u(t))dx^{i}(t)=0$. It follows the multitime adjoint Pfaff system

[TABLE]

whose solution $p(t)=(p^{\alpha}(t))$ is called the costate vector. Here, the symbol $d$ in the left hand member of the adjoint equation means the differentiation with respect to $p$ and $x$.

We use the Lagrangian $m$-form

[TABLE]

Introducing the $(m-1)$-forms

[TABLE]

a costate variable vector or Lagrange multiplier vector $p=p^{\alpha}(t)\displaystyle\frac{\partial}{\partial t^{\alpha}}$ is identified to the $(m-1)$-form $p=p^{\lambda}(t)\omega_{\lambda}$. We build a Hamiltonian $m$-form

[TABLE]

Theorem (Multitime maximum principle) Suppose that the problem of maximizing the functional (11) constrained by (12) has an interior optimal solution $\hat{u}(t)$, which determines the optimal evolution $x(t)$. Then there exists a costate function $(p(t))$ such that

[TABLE]

the function $(p(t))$ is the unique solution of the following Pfaff system (adjoint system)

[TABLE]

and satisfies the critical point conditions

[TABLE]

Proof We use the Hamiltonian $m$-form ${\cal H}$. The solutions of the foregoing problem are among the solutions of the free maximization problem of the functional

[TABLE]

where $\tilde{\Omega}=(\Omega_{t_{0}t_{1}},x(\Omega_{t_{0}t_{1}}))\subset R^{m}_{+}\times R^{n}$.

Suppose that there exists a continuous control $\hat{u}(t)$ defined over the interval $\Omega_{t_{0}t_{1}}$ with $\hat{u}(t)\in\hbox{Int}\,{U}$ which is an optimum point in the previous problem. Now we consider a control variation $u(t,\epsilon)=\hat{u}(t)+\epsilon h(t)$, where $h$ is an arbitrary continuous vector function, and a state variation $x(t,\epsilon),\,t\in\Omega_{t_{0}t_{1}}$, connected by

[TABLE]

Since $\hat{u}(t)\in\hbox{Int}\,{\cal U}$ and any continuous function over a compact set $\Omega_{t_{0}t_{1}}$ is bounded, there exists $\epsilon_{h}>0$ such that $u(t,\epsilon)=\hat{u}(t)+\epsilon h(t)\in\hbox{Int}\,{U},\,\,\forall|\epsilon|<\epsilon_{h}$. This $\epsilon$ is used in our variational arguments.

For $|\epsilon|<\epsilon_{h}$, we define the function

[TABLE]

[TABLE]

Differentiating with respect to $\epsilon$, it follows

[TABLE]

[TABLE]

[TABLE]

[TABLE]

[TABLE]

Evaluating at $\epsilon=0$, we find $\tilde{\Omega}(0)=\tilde{\Omega}$ and

[TABLE]

[TABLE]

[TABLE]

where $x(t)$ is the $m$-sheet of the state variable corresponding to the optimal control $\hat{u}(t)$.

To evaluate the multiple integral

[TABLE]

we integrate by parts, via the formula

[TABLE]

obtaining

[TABLE]

where the symbol $d$ in the last integral means the differentiation with respect to $p$ and $x$. Now we apply the Stokes integral formula

[TABLE]

where $(n^{\beta}(t))$ is the unit normal vector to the boundary ${\partial{\tilde{\Omega}}}$. Since the integral from the middle can be written

[TABLE]

we find $J^{\prime}(0)$ as

[TABLE]

[TABLE]

[TABLE]

[TABLE]

We select the costate function $p(t)$ as solution of the adjoint Pfaff equation (boundary value problem)

[TABLE]

[TABLE]

On the other hand, we need $J^{\prime}(0)=0$ for all $h(t)=(h^{a}(t))$. Since the variation $h$ is arbitrary, we get (critical point condition)

[TABLE]

The foregoing equations (14) and (15) can be written

[TABLE]

Let us start with a

1.4.2 Multitime optimal control problem of

maximizing a curvilinear integral functional

Find

[TABLE]

subject to

[TABLE]

It is supposed that $L_{\alpha}:\Omega_{t_{0}t_{1}}\times A\times U\to R$ and $a_{i}:A\times U\to R,\,i=1,...,n$ are $C^{2}$ functions. Ingredients: ${\cal L}=L_{\alpha}(t,x(t),u(t))dt^{\alpha}$ is an $1$-form, $A$ is a bounded and closed subset of $R^{n}$, containing the images of the $m$-sheets $x(t),\,t\in\Omega_{t_{0}t_{1}}$ of the controlled system, and $x_{0}$ and $x_{1}$ are the initial and final states of the $m$-sheet $x(t)$ in the controlled system. The set, in which the control functions $u^{a}$ takes their values, is called as $U$, which is a bounded and closed subset of $R^{k}$.

Let us find the first order necessary conditions for an optimal pair $(x,u)$. We fix the control $u(t)$ and variate the state $x(t)$ into $x(t,\epsilon),\,\epsilon=(\epsilon^{1},...,\epsilon^{m})$. We find the multitime infinitesimal deformation (Pfaff) system

[TABLE]

of the nonholonomic constraint $a_{i}(x(t),u(t))dx^{i}(t)=0$. It follows the multitime adjoint Pfaff system

[TABLE]

whose solution $p(t)$ is called the costate vector. Here, the symbol $d$ in the left hand member of the adjoint equation means the differentiation with respect to $p$ and $x$.

We use the Lagrangian $1$-form

[TABLE]

Introducing a costate variable or Lagrange multiplier $p$, we build a Hamiltonian $1$-form

[TABLE]

Theorem (Multitime maximum principle) Suppose that the problem of maximizing the functional (16) constrained by (17) has an interior optimal solution $\hat{u}(t)$, which determines the optimal evolution $x(t)$. Then there exists a costate function $(p(t))$ such that

[TABLE]

the function $(p(t))$ is the unique solution of the following Pfaff system (adjoint system)

[TABLE]

and the critical point conditions

[TABLE]

hold.

Proof We use the Hamiltonian $1$-form ${\cal H}$. The solutions of the foregoing problem are between the solutions of the free maximization problem of the curvilinear integral functional

[TABLE]

where $\tilde{\Gamma}=(\Gamma_{t_{0}t_{1}},x(\Gamma_{t_{0}t_{1}}))\subset R^{m}_{+}\times R^{n}$.

Suppose that there exists a continuous control $\hat{u}(t)$ defined over the interval $\Omega_{t_{0}t_{1}}$ with $\hat{u}(t)\in\hbox{Int}\,{U}$ which is an optimum point in the previous problem. We consider a control variation $u(t,\epsilon)=\hat{u}(t)+\epsilon h(t)$, where $h$ is an arbitrary continuous vector function, and a state variation $x(t,\epsilon),\,t\in\Omega_{t_{0}t_{1}}$, connected by

[TABLE]

Since $\hat{u}(t)\in\hbox{Int}\,{\cal U}$ and a continuous function over a compact set $\Omega_{t_{0}t_{1}}$ is bounded, there exists $\epsilon_{h}>0$ such that $u(t,\epsilon)=\hat{u}(t)+\epsilon h(t)\in\hbox{Int}\,{U},\,\,\forall|\epsilon|<\epsilon_{h}$. This $\epsilon$ is used in our variational arguments.

For $|\epsilon|<\epsilon_{h}$, we define the function

[TABLE]

[TABLE]

Differentiating with respect to $\epsilon$, it follows

[TABLE]

[TABLE]

[TABLE]

[TABLE]

[TABLE]

Evaluating at $\epsilon=0$, we find $\tilde{\Gamma}(0)=\tilde{\Gamma}$ and

[TABLE]

[TABLE]

[TABLE]

where $x(t)$ is the $m$-sheet of the state variable corresponding to the optimal control $\hat{u}(t)$.

To evaluate the curvilinear integral

[TABLE]

we integrate by parts, via the formula

[TABLE]

obtaining

[TABLE]

[TABLE]

where the symbol $d$ in the last integral means the differentiation with respect to $p$ and $x$. We find

[TABLE]

[TABLE]

[TABLE]

[TABLE]

We select the costate function $p(t)$ as solution of the adjoint Pfaff equation (terminal value problem)

[TABLE]

subject to $p(t_{1})a_{i}(x(t_{1}),\hat{u}(t_{1}))+g_{x^{i}}(x(t_{1}))=0$. On the other hand, we need $J^{\prime}(0)=0$ for all $h(t)=(h^{a}(t))$. Since the variation $h$ is arbitrary, we get (critical point condition)

[TABLE]

The foregoing equations (19) and (20) can be written

[TABLE]

2 Optimal control on distributions

described by vector fields

2.1 Infinitesimal deformations and adjointness on

distributions

The same distribution $D$ can be described in terms of smooth vector fields (or generators),

[TABLE]

if and only if $n\geq 3$. Any vector field $Y$ in $D$ can be written in the form $Y(x)=u^{a}(x)X_{a}(x)$.

Let $x(t)$ be a curve solution of the differential system

[TABLE]

Let $x(t;\epsilon)$ be a differentiable variation of $x(t)$, i.e.,

[TABLE]

Denoting $y^{i}(t)=\displaystyle\frac{\partial x^{i}}{\partial\epsilon}(t;0)$, we find the single-time infinitesimal deformation system

[TABLE]

The single-time adjoint (dual) system is

[TABLE]

whose solution $p=(p_{k})$ is called the costate vector. The foregoing PDE systems (17) and (18) are adjoint (dual) in the sense of constant interior product of solutions, i.e., the scalar product $p_{k}\,y^{k}$ is a first integral.

Let $x(t)$ be an $m$-sheet integral submanifold of the distribution $D$, i.e., a solution of the multitime partial differential system

[TABLE]

Let $\epsilon=(\epsilon^{\alpha}),\,\alpha=1,...,m$ and let $x(t;\epsilon)$ be a differentiable variation of $x(t)$, i.e.,

[TABLE]

Introducing the vector fields $y^{i}_{\alpha}(t)=\displaystyle\frac{\partial x^{i}}{\partial\epsilon^{\alpha}}(t;0)$, we find the multitime infinitesimal deformation system

[TABLE]

The multitime adjoint (dual) system is

[TABLE]

whose solution $p=(p^{\alpha}_{k})$ is called the costate matrix. The foregoing PDE systems (19) and (20) are adjoint (dual) in the sense of constant interior product of solutions, i.e., the scalar product $p^{\alpha}_{k}\,y^{k}_{\alpha}$ is a first integral.

Of course, taking the trace, we can define the costate matrix $p:\Omega_{0T}\to R^{mn},\,\,p=(p^{\alpha}_{k}),$ as the solution of the divergence adjoint PDE system (trace)

[TABLE]

But than, the PDEs systems (19) and (20) are adjoint (dual) in the sense of zero total divergence of the tensor field $Q^{\alpha}_{\beta}=p^{\alpha}_{k}\,y^{k}_{\beta}$ produced by their solutions. The divergence dual PDE system (21) has solutions since it contains $n$ PDEs with $nm$ unknown functions $p^{\alpha}_{i}$. We can select a solution of the gradient form $p^{\alpha}_{k}(t)=\displaystyle\frac{\partial v^{\alpha}}{\partial x^{k}}(t,x(t))$.

Remark The multitime adjoint Pfaff system can be defined independent on the dimension of the parameter $\epsilon$. Particularly, the multitime adjoint Pfaff system can be

[TABLE]

2.2 Single-time optimal control problems on

a distribution

Let

[TABLE]

be a distribution on $R^{n}$ and $x(t),\,\,t\in I=[t_{0},t_{1}],$ be an integral curve of the driftless control system

[TABLE]

A single-time optimal control problem is defined to be maximizing the functional

[TABLE]

subject to

[TABLE]

It is supposed that $L:I\times A\times U\to R$ is a $C^{2}$ function and $X_{a}:A\to R,\,a=1,...,n-1$ are $C^{2}$ functions. Ingredients: $A$ is a bounded and closed subset of $R^{n}$, which the trajectory of controlled system is constrained to stay for $t\in I$, and $x_{0}$ and $x_{1}$ are the initial and final states of the trajectory $x(t)$ in the controlled system. The set in which the control functions $u^{a}$ takes their values in it, is called as $U$, which is a bounded and closed subset of $R^{n-1}$. The map $u$ is assumed to be piecewise smooth or piecewise analytic. Such maps are called admissible and the space ${\cal U}$ of all such maps is called the set of admissible controls.

Let us find the first order necessary conditions for an optimal pair $(x,u)$. Firstly, the single-time infinitesimal deformation (Pfaff) equation of the constraint $dx(t)=u^{a}(x(t))X_{a}(x(t))dt$ is the system (17) the single-time adjoint Pfaff equation is the system (18).

The control variables may be open-loop $u^{a}(t)$, depending directly on the time variable $t$, or closed-loop (or feedback) $u^{a}(x(t))$, depending on the state $x(t)$.

Open-loop control variables

To simplify, we accept an open-loop control $u^{a}(t)$. Using the Lagrangian $1$-form

[TABLE]

we build the Hamiltonian $1$-form

[TABLE]

Theorem (Single-time maximum principle) Suppose that the problem of maximizing the functional (22) constrained by (23) has an interior optimal solution $\hat{u}(t)$, which determines the optimal evolution $x(t)$. Then there exists a costate vector p(t) = $(p_{i}(t))$ such that

[TABLE]

the function $p(t)$ is the unique solution of the following Pfaff system (adjoint system)

[TABLE]

and the critical point conditions

[TABLE]

hold.

Proof We use the Lagrangian $1$-form ${\cal L}$. The solutions of the forgoing problem are between the solutions of the free maximization problem of the curvilinear integral functional

[TABLE]

where $\tilde{\Gamma}=([t_{0},t_{1}],x([t_{0},t_{1}]))\subset R_{+}\times R^{n}$.

Suppose that there exists a continuous control $\hat{u}(t)$ defined over the interval $I$ with $\hat{u}(t)\in\hbox{Int}\,{U}$ which is an optimum point in the previous problem. Now consider a variation $u(t,\epsilon)=\hat{u}(t)+\epsilon h(t)$, where $h$ is an arbitrary continuous vector function. Since $\hat{u}(t)\in\hbox{Int}\,{\cal U}$ and a continuous function over a compact set $I$ is bounded, there exists $\epsilon_{h}>0$ such that $u(t,\epsilon)=\hat{u}(t)+\epsilon h(t)\in\hbox{Int}\,{U},\,\,\forall|\epsilon|<\epsilon_{h}$. This $\epsilon$ is used in our variational arguments.

Define $x(t,\epsilon)$ as the $1$-sheet of the state variable corresponding to the control variable $u(t,\epsilon)$, i.e.,

[TABLE]

and $x(t_{0},0)=x_{0}$, $x(t_{1},0)=x_{1}$. For $|\epsilon|<\epsilon_{h}$, we define the function

[TABLE]

[TABLE]

Differentiating with respect to $\epsilon$, it follows

[TABLE]

[TABLE]

[TABLE]

Evaluating at $\epsilon=0$, we find $\tilde{\Gamma}(0)=\tilde{\Gamma}$ and

[TABLE]

[TABLE]

[TABLE]

where $x(t)$ is the curve of the state variable corresponding to the optimal control $\hat{u}(t)$. Since the integral from the middle can be written

[TABLE]

we find $J^{\prime}(0)$ as

[TABLE]

[TABLE]

[TABLE]

We select the costate function $p(t)$ as solution of the adjoint Pfaff equation (boundary value problem)

[TABLE]

On the other hand, we need $J^{\prime}(0)=0$ for all $h(t)=(h^{a}(t))$. Since the variation $h$ is arbitrary, we get (critical point condition)

[TABLE]

Example Consider the ODE system $\dot{x}^{1}(t)={x^{2}}^{2}(t),\dot{x}^{2}(t)=u(t)$ generated by the vector fields $X={x^{2}}^{2}\frac{\partial}{\partial x^{1}}$, $Y=\frac{\partial}{\partial x^{2}}$. We compute the Lie brackets

[TABLE]

The vector fields $Y$ and $[[X,Y],Y]$ are linearly independent. On the other hand, the $x^{1}$-coordinate is increasing since ${x^{2}}^{2}\geq 0$. Consequently, the system is not really controllable.

Closed-loop control variables

Now, we accept a closed-loop control $u^{a}(x(t))$. Using the Lagrangian $1$-form

[TABLE]

we build the Hamiltonian $1$-form

[TABLE]

Theorem (Single-time maximum principle) Suppose that the problem of maximizing the functional (22) constrained by (23) has an interior optimal solution $\hat{u}(x(t))$, which determines the optimal evolution $x(t)$. Then there exists a costate vector p(t) = $(p_{i}(t))$ such that

[TABLE]

the function $p(t)$ is the unique solution of the following Pfaff system (adjoint system)

[TABLE]

and the critical point conditions

[TABLE]

hold.

Proof The new functional is

[TABLE]

A variation $x(t,\epsilon)$ induces a variation $u(x(t,\epsilon))=\hat{u}(x(t))+\epsilon h(t)$. Then

[TABLE]

It follows

[TABLE]

[TABLE]

[TABLE]

[TABLE]

Evaluating at $\epsilon=0$, we find $\tilde{\Gamma}(0)=\tilde{\Gamma}$ and

[TABLE]

[TABLE]

[TABLE]

[TABLE]

Since the integral from the middle can be written

[TABLE]

we find $J^{\prime}(0)$ as

[TABLE]

[TABLE]

[TABLE]

We select the costate function $p(t)$ as solution of the adjoint Pfaff equation (boundary value problem)

[TABLE]

On the other hand, we need $J^{\prime}(0)=0$ for all $h(t)=(h^{a}(t))$. Since the variation $h$ is arbitrary, we get (critical point condition)

[TABLE]

2.3 Multitime optimal control problems on

a distribution

Let

[TABLE]

$n\geq 3$, be a distribution on $R^{n}$ and $x(t),\,\,t\in\Omega_{t_{0}t_{1}}$, be an $m$-sheet of the driftless control system

[TABLE]

Let us start with a

2.3.1 Multitime optimal control problem of

maximizing a multiple integral functional

Find

[TABLE]

subject to

[TABLE]

It is supposed that $L:\Omega_{t_{0}t_{1}}\times A\times U\to R$ is a $C^{2}$ function and $u^{a}:A\to R^{n-1},\,a=1,...,n-1$, $X^{i}_{a}:A\to R^{(n-1)n},\,a=1,...,n-1,i=1,...,n$ are $C^{2}$ functions. Ingredients: $\omega=dt^{1}\cdots dt^{m}$ is the volume element, $A$ is a bounded and closed subset of $R^{n}$, which the $m$-sheet of controlled system is constrained to stay for $t\in\Omega_{t_{0}t_{1}}$, and $x_{0}$ and $x_{1}$ are the initial and final states of the $m$-sheet $x(t)$ in the controlled system. The set in which the control functions $u^{a}(t)$ takes their values in it, is called as $U$, which is a bounded and closed subset of $R^{n-1}$.

Let us find the first order necessary conditions for an optimal pair $(x,u)$. Firstly, the multitime infinitesimal deformation (Pfaff) system of the constraint (28) is (19), and the multitime adjoint Pfaff system is (20).

The control variables may be open-loop $u^{a}(t)$, depending directly on the multitime variable $t$, or closed-loop (or feedback) $u^{a}(x(t))$, depending on the state $x(t)$.

To simplify, we accept an open-loop control. Introducing the $(m-1)$-forms

[TABLE]

a costate variable matrix or Lagrange multiplier matrix $p=p^{\alpha}_{i}(t)\displaystyle\frac{\partial}{\partial t^{\alpha}}\otimes dx^{i}$ is identified to the $(m-1)$-forms $p_{i}=p^{\lambda}_{i}(t)\omega_{\lambda}$. We use the Lagrangian $m$-form

[TABLE]

and the Hamiltonian $m$-form

[TABLE]

Theorem (Multitime maximum principle) Suppose that the problem of maximizing the functional (27) constrained by (28) has an interior optimal solution $\hat{u}(t)$, which determines the optimal evolution $x(t)$. Then there exists a costate matrix $(p(t))$ such that

[TABLE]

the function $(p(t))$ is the unique solution of the following Pfaff system (adjoint system)

[TABLE]

and the critical point conditions

[TABLE]

hold.

Proof We use the Lagrangian $m$-form ${\cal L}$. The solutions of the foregoing problem are between the solutions of the free maximization problem of the functional

[TABLE]

where $\tilde{\Omega}=(\Omega_{t_{0}t_{1}},x(\Omega_{t_{0}t_{1}}))\subset R^{m}_{+}\times R^{n}$.

Suppose that there exists a continuous control $\hat{u}(t)$ defined over the interval $\Omega_{t_{0}t_{1}}$ with $\hat{u}(t)\in\hbox{Int}\,{U}$ which is an optimum point in the previous problem. Now consider a variation $u(t,\epsilon)=\hat{u}(t)+\epsilon h(t)$, where $h$ is an arbitrary continuous vector function. Since $\hat{u}(t)\in\hbox{Int}\,{\cal U}$ and a continuous function over a compact set $\Omega_{t_{0}t_{1}}$ is bounded, there exists $\epsilon_{h}>0$ such that $u(t,\epsilon)=\hat{u}(t)+\epsilon h(t)\in\hbox{Int}\,{U},\,\,\forall|\epsilon|<\epsilon_{h}$. This $\epsilon$ is used in our variational arguments.

Define $x(t,\epsilon)$ as the $m$-sheet of the state variable corresponding to the control variable $u(t,\epsilon)$, i.e.,

[TABLE]

and $x(t_{0},0)=x_{0}$, $x(t_{1},0)=x_{1}$. For $|\epsilon|<\epsilon_{h}$, we define the function

[TABLE]

[TABLE]

Differentiating with respect to $\epsilon$, it follows

[TABLE]

[TABLE]

[TABLE]

Evaluating at $\epsilon=0$, we find

[TABLE]

[TABLE]

[TABLE]

where $x(t)$ is the $m$-sheet of the state variable corresponding to the optimal control $\hat{u}(t)$.

To evaluate the multiple integral

[TABLE]

we integrate by parts, via the formula

[TABLE]

obtaining

[TABLE]

Now we apply the Stokes integral formula

[TABLE]

where $(n^{\beta}(t))$ is the unit normal vector to the boundary ${\partial{\tilde{\Omega}}}$. Since the integral from the middle can be written

[TABLE]

we find $J^{\prime}(0)$ as

[TABLE]

[TABLE]

[TABLE]

We select the costate function $p(t)$ as solution of the adjoint Pfaff equation (boundary value problem)

[TABLE]

On the other hand, we need $J^{\prime}(0)=0$ for all $h(t)=(h^{a}(t))$. Since the variation $h$ is arbitrary, we get (critical point condition)

[TABLE]

Let us start with a

2.3.2 Multitime optimal control problem of

maximizing a curvilinear integral functional

Find

[TABLE]

subject to

[TABLE]

It is supposed that $L_{\alpha}:\Omega_{t_{0}t_{1}}\times A\times U\to R$ and $u^{a}:A\to R^{n-1},\,a=1,...,n-1$, $X^{i}_{a}:A\to R^{(n-1)n},\,a=1,...,n-1,i=1,...,n$ are $C^{2}$ functions. Ingredients: ${\cal L}=L_{\alpha}(t,x(t),u(t))dt^{\alpha}$ is an $1$-form, $A$ is a bounded and closed subset of $R^{n}$, which the $m$-sheet of controlled system is constrained to stay for $t\in\Omega_{t_{0}t_{1}}$, and $x_{0}$ and $x_{1}$ are the initial and final states of the $m$-sheet $x(t)$ in the controlled system. The set in which the control functions $u^{a}$ takes their values in it, is called as $U$, which is a bounded and closed subset of $R^{n-1}$.

Let us find the first order necessary conditions for an optimal pair $(x,u)$. Firstly, the multitime infinitesimal deformation (Pfaff) system of the constraint (33) is (19), and the multitime adjoint Pfaff system is (*).

The control variables may be open-loop $u^{a}(t)$, depending directly on the multitime variable $t$, or closed loop (or feedback) $u^{a}(x(t))$, depending on the state $x(t)$.

To simplify, we accept an open loop control. Introducing a costate variable vector or Lagrange multiplier $p=(p_{i}(t))$, we build a Lagrangian $1$-form

[TABLE]

and a Hamiltonian $1$-form

[TABLE]

Theorem (Multitime maximum principle) Suppose that the problem of maximizing the functional (32) constrained by (33) has an interior optimal solution $\hat{u}(t)$, which determines the optimal evolution $x(t)$. Then there exists a costate function $(p(t))$ such that

[TABLE]

the function $(p(t))$ is the unique solution of the following Pfaff system (adjoint system)

[TABLE]

and the critical point conditions

[TABLE]

hold.

Proof We use the Lagrangian $1$-form ${\cal L}$. The solutions of the foregoing problem are between the solutions of the free maximization problem of the curvilinear integral functional

[TABLE]

where $\tilde{\Gamma}=(\Gamma_{t_{0}t_{1}},x(\Gamma_{t_{0}t_{1}}))\subset R^{m}_{+}\times R^{n}$.

Suppose that there exists a continuous control $\hat{u}(t)$ defined over the interval $\Omega_{t_{0}t_{1}}$ with $\hat{u}(t)\in\hbox{Int}\,{U}$ which is an optimum point in the previous problem. Now consider a variation $u(t,\epsilon)=\hat{u}(t)+\epsilon h(t)$, where $h$ is an arbitrary continuous vector function. Since $\hat{u}(t)\in\hbox{Int}\,{\cal U}$ and a continuous function over a compact set $\Omega_{t_{0}t_{1}}$ is bounded, there exists $\epsilon_{h}>0$ such that $u(t,\epsilon)=\hat{u}(t)+\epsilon h(t)\in\hbox{Int}\,{U},\,\,\forall|\epsilon|<\epsilon_{h}$. This $\epsilon$ is used in our variational arguments.

Define $x(t,\epsilon)$ as the $m$-sheet of the state variable corresponding to the control variable $u(t,\epsilon)$, i.e.,

[TABLE]

and $x(t_{0},0)=x_{0}$, $x(t_{1},0)=x_{1}$. For $|\epsilon|<\epsilon_{h}$, we define the function

[TABLE]

[TABLE]

Differentiating with respect to $\epsilon$, it follows

[TABLE]

[TABLE]

[TABLE]

Evaluating at $\epsilon=0$, we find $\tilde{\Gamma}(0)=\tilde{\Gamma}$ and

[TABLE]

[TABLE]

[TABLE]

where $x(t)$ is the $m$-sheet of the state variable corresponding to the optimal control $\hat{u}(t)$.

To evaluate the curvilinear integral

[TABLE]

we integrate by parts, via the formula

[TABLE]

obtaining

[TABLE]

We find

[TABLE]

[TABLE]

[TABLE]

We select the costate function $p(t)$ as solution of the adjoint Pfaff equation (boundary value problem)

[TABLE]

On the other hand, we need $J^{\prime}(0)=0$ for all $h(t)=(h^{a}_{\alpha}(t))$. Since the variation $h$ is arbitrary, we get (critical point condition)

[TABLE]

Example: Nonholonomic control of torsion of a cylinder or prism

Suppose the torsion of a cylinder or prism is described by the controlled Pfaff equation

[TABLE]

where the control $(u,v)$ is not subject to constraints. If the complete integrability condition $\displaystyle{\partial v\over\partial x}-\displaystyle{\partial u\over\partial y}=2$ is verified identically, then we have a holonomic evolution. Otherwise, we have a nonholonomic evolution. Using the controlled Pfaff equation as constraint, we want to minimize the functional

[TABLE]

where $\Gamma$ is a $C^{1}$ curve joining the points $(0,0)$ and $(x_{0},y_{0})$, and $c_{1},c_{2}$ are constants. The minimization of the previous integral is equivalent to the maximization of the cost functional

[TABLE]

subject to controlled Pfaff equation.

Let us find the optimal manifold (surface or curve) of evolution, using the two-variable maximum principle theory. For that we introduce the 1-forms:

[TABLE]

Taking $p_{0}=1$, we obtain

[TABLE]

The adjoint equation $dp=-\displaystyle{\partial\over\partial z}\eta=c_{1}dx+c_{2}dy,\,\,p(0,0)=0$ has the solution $p(x,y)=c_{1}x+c_{2}y$. The maximization condition

[TABLE]

[TABLE]

gives the optimal law

[TABLE]

Replacing in the evolution Pfaff equation we obtain

[TABLE]

1. If the complete integrability condition

[TABLE]

i.e., $4c_{1}c_{2}+c_{2}^{2}-c_{1}^{2}=0$ is satisfied, then the evolution surface is

[TABLE]

2. If $4c_{1}c_{2}+c_{2}^{2}-c_{1}^{2}\neq 0$, then the Pfaff evolution equation admits only solutions which are curves (nonholonomic surface in $R^{3}$): $x=x(t),y=y(t),z=z(t),t\in I,$ with

[TABLE]

In this case, for determining $z=z(t)$, we must take $x=x(t),y=y(t),\,t\in I$ as a parametrization of the curve $\Gamma$ from the cost functional. In fact, the problem is reduced to optimization of a simple integral constrained by a differential equation.

3 Curvilinear integral functionals

depending on curves

Let $\Omega_{t_{0}t_{1}}\subset R^{m}$, let ${\Gamma_{t_{0}t_{1}}}\subset\Omega_{t_{0}t_{1}}$ be a $C^{1}$ curve and

[TABLE]

be a curvilinear integral functional depending on the curve ${\Gamma_{t_{0}t_{1}}}$. Consider a variation ${\Gamma_{t_{0}t_{1}}(\epsilon)}:t=(t^{\alpha}(\tau,\epsilon))$ of the curve ${\Gamma_{t_{0}t_{1}}}:t=(t^{\alpha}(\tau))$, with the same endpoints. Suppose $L_{\alpha}(t,x(t),x_{\gamma}(t))dt^{\alpha}$ is stationary with respect to $\epsilon$. Then

[TABLE]

The closed curve $C=\Gamma_{t_{0}t_{1}}(\epsilon)\cup{\Gamma_{t_{1}t_{0}}(0)}$ is the boundary of a surface $S$. We evaluate $J(\epsilon)-J(0)$, using Stokes formula,

[TABLE]

[TABLE]

[TABLE]

[TABLE]

Now we use the variation vector field $\frac{\partial t^{\alpha}}{\partial\epsilon}(\tau,\epsilon)|_{\epsilon=0}=\xi^{\alpha}(\tau)$. Replacing $dt^{\alpha}=\epsilon\,\xi^{\alpha}$, the surface integral is transformed to a curvilinear integral

[TABLE]

It follows

[TABLE]

Suppose ${\Gamma_{t_{0}t_{1}}}$ is a critical point of the functional, hence $J^{\prime}(0)=0,\,\forall\xi$. Consequently

[TABLE]

If the curvilinear integral is path independent, then this relation is identically satisfied. If the curvilinear integral is path dependent, then the discussion depends on $m$ since $a_{\alpha\beta}=D_{\alpha}L_{\beta}-D_{\beta}L_{\alpha}$ is an anti-symmetric matrix, and consequently its determinant $d=\det(a_{\alpha\beta})$ is either [math], for $m$ odd, or $\geq 0$, for $m$ even. For $m$-odd we have solutions, i.e., critical curves; for $m$-even, we have either no solution for $d>0$ or solutions for $d=0$. Since the differential system is of order one, the curve solution is determined only by a single condition (the general bilocal problems have no solution). The extremum problems have sense only if we add supplimentary conditions (an initial condition + an isoperimetric condition).

Variant Let $\Omega_{t_{0}t_{1}}\subset R^{m}$, let ${\Gamma_{t_{0}t_{1}}}\subset\Omega_{t_{0}t_{1}}$ be a $C^{1}$ curve and

[TABLE]

be a curvilinear integral functional depending on the curve ${\Gamma_{t_{0}t_{1}}}$. Consider a variation ${\Gamma_{t_{0}t_{1}}(\epsilon)}:t=(t^{\alpha}(\tau,\epsilon))$ of the curve ${\Gamma_{t_{0}t_{1}}}:t=(t^{\alpha}(\tau))$, with the same endpoints. Denote $M_{\alpha}(t)=L_{\alpha}(t,x(t),x_{\gamma}(t))$.

Then

[TABLE]

To compute $J^{\prime}(0)$, we use the variation vector field $\frac{\partial t^{\alpha}}{\partial\epsilon}(\tau,\epsilon)|_{\epsilon=0}=\xi^{\alpha}(\tau)$. From

[TABLE]

we obtain

[TABLE]

Integrating by parts, we find

[TABLE]

Remark The variation of the function $x(t)$ has nothing to do with the variation of the curve.

4 Optimization of mechanical work on

Riemannian manifolds

Let $(M,g)$ be a Riemannian manifold and $X$ a $C^{2}$ vector field on M. Let $x=(x^{1},...,x^{n})$ denote the local coordinates relative to a fixed local map $(V,h)$. Since $h:V\rightarrow R^{n}$ is a diffeomorphism, we denote by $\Omega_{x_{0}x_{1}}$ a subset of $V$ diffeomorphic through $h$ with the hyper-parallelepiped in $R^{n}$ having $h(x_{0})$ and $h(x_{1})$ as diagonal points.

Let ${\Gamma_{x_{0}x_{1}}}:x^{i}=x^{i}(t),t\in[t_{0},t_{1}]$ be an arbitrary $C^{1}$ curve on $M$ which joins the points $x(t_{0})=x_{0},x(t_{1})=x_{1}.$ The functional

[TABLE]

is generated by the mechanical work produced by the force $\omega_{j}=g_{ij}(x)X^{i}(x)$ along the curve $\Gamma_{x_{0}x_{1}}$.

Let $X$ be a nowhere zero vector field. $X$ is called a geodesic vector field iff $\nabla_{X}X=0$. Thus $X$ is geodesic iff each of its integral curves is a geodesic.

Theorem If $X$ is a unit geodesic vector field and $\gamma_{x_{0}x_{1}}$ is a field line, then the curve $\gamma_{x_{0}x_{1}}$ is a maximum point of the functional $J(\Gamma_{x_{0}x_{1}})$ and the maximum value is the length of $\gamma_{x_{0}x_{1}}$.

Proof Let us find

[TABLE]

where

[TABLE]

The critical point condition, with respect to the curve ${\Gamma_{x_{0}x_{1}}}$, is

[TABLE]

It is identically satisfied, because ${\gamma_{x_{0}x_{1}}}$ is a field line, the geodesic condition implies $\nabla_{X}X^{i}=0$ and the the condition of unit vector field gives $g_{ij}X^{j}\nabla_{k}X^{i}=0.$

On the other hand, the inequality

[TABLE]

becomes an equality if $dx^{j}=X^{j}(x(t))dt$, i.e., $x(t)$ is a field line of $X(x)$. Under the condition $||X||=1$, the maximum value of the foregoing functional is the length of ${\gamma_{x_{0}x_{1}}}$.

5 Bang-bang control on distributions

The same distribution $D$ can be described in terms of vector fields,

[TABLE]

Bang-bang control is an optimal or suboptimal piecewise constant control whose values are defined by bounds imposed on the amplitude of control components. The control changes its values according to the switching function which may be found using the maximum principle. The discontinuity of the bang-bang control leads to discontinuity of a value function for the considered optimal control problem. Typical problems with bang-bang optimal control include time and terminal cost optimal control for linear control systems. Bang-bang optimization offers a direct explanation for an otherwise perplexing observation and indicates that evolution is operating according to principles that every engineer knows. The balck hole applications covered in this Section refer to the controllability of the ODE or PDE system by bang-bang controls.

5.1 Single-time bang-bang optimal control

Let $x(t),\,\,t\in I=[0,\tau]\subset R$, be an integral curve of the distribution $D$. Any curve in the distribution $\Delta=\hbox{span}\{X_{a},a=1,...,n-1\}$ is a solution of the controlled ODE system

[TABLE]

called driftless control system.

(1) Time minimum problem Let $U=[-1,1]^{n-1}\subset R^{n-1}$ be the control set. Giving the starting point $x_{0}\in R^{n}$, find an optimal control $u^{*}(\cdot)$ such that

[TABLE]

using (ODE) evolution as constraint. Since $\tau^{*}=I(u^{*}(\cdot))$, the optimal point $\tau^{*}$ ensures the minimum time to steer to the origin. This time optimum problem is equivalent to a controllability one.

Solution To prove the existence of a bang-bang control, we use the single-time Pontryaguin Maximum Principle. The Hamiltonian $H(x,p,u)=-1+p_{i}X^{i}_{a}(x)u^{a}$ gives the adjoint ODE system $\dot{p}_{j}(t)=-p_{i}(t)\frac{\partial X^{i}_{a}}{\partial x^{j}}(x(t))u^{a}(t)$. The extremum of the linear function $u\to H$ exists since each control variable belong to the interval $[-1,1]$; for optimum, the control must be at a vertex of $\partial U$ (see, linear optimization, simplex method). If $Q_{a}(t)=p_{i}(t)X^{i}_{a}(x(t))$, then the optimal control $u^{*a}$ must be the function (bang-bang control)

[TABLE]

Suppose the Lebesgue measure of each set $\{t\in[0,\tau]:Q_{a}(t)=0\}$ vanishes. Then the singular control is ruled out and the remaining possibilities are bang-bang controls. This optimal control is discontinuous since each component jumps from a minimum to a maximum and vice versa in response to each change in the sign of each $Q_{a}(t)$. The functions $Q_{a}(t)$ are called switching functions.

(2) Optimal terminal value Let $U=[-1,1]^{n-1}\subset R^{n-1}$ be the control set. Suppose we have to

Minimize the terminal cost functional

[TABLE]

subject to the driftless control system

[TABLE]

Solution Since the control Hamiltonian $H(x,p,u)=p_{i}X^{i}_{a}(x)u^{a}$ is linear in the control, the optimal control is a bang-bang. Automatically we find the optimal costate function and the optimal evolution.

5.2 Multitime bang-bang optimal control

Let $\Omega_{0\tau}$ be the hyperparallelipiped determined by two opposite diagonal points $0=(0,...,0)$ and $\tau=(\tau^{1},...,\tau^{m})$ in $R^{m}_{+}$, endowed with the product order. Let $x(t),\,\,t\in\Omega_{0\tau}\subset R^{m}_{+}$, be an integral $m$-sheet of the distribution $D$, i.e., a solution of a multitime piecewise completely integrable PDE system

[TABLE]

This sort of controlled PDE is called a driftless control system. Of course, the piecewise complete integrability conditions

[TABLE]

restrict the controls, excepting the case when they are identically satisfied.

To show that the driftless control system is multitime controllable, by bang-bang controls (see also [10]), we use the next multitime minimum problems

Case of multiple integral functional Let $U=[-1,1]^{m(n-1)}\subset R^{m(n-1)}$ be the control set. Giving the starting point $x_{0}\in R^{n}$, find an optimal control $u^{*}(\cdot)$ such that

[TABLE]

using a completely integrable two-time evolution (PDE) as constraint and supposing that (CIC) are identically satisfied. Since $\tau^{*1}\cdots\tau^{*2}=I(u^{*}(\cdot))$, the optimal point $\tau^{*}=(\tau^{*1},...,\tau^{*m})$ ensures the minimum multitime ”volume” to steer to the origin. This two-time optimum problem consists in devising a control such that to transfer a given initial state to a specified target (controllability problem).

Solution We apply the multitime maximum principle which proves the existence of a bang-bang control. The Hamiltonian $H(x,p,u)=-1+p_{i}^{\alpha}X^{i}_{a}(x)u^{a}_{\alpha}$ gives the adjoint PDE system $\frac{\partial p^{\alpha}_{j}}{\partial t^{\alpha}}(t)=-p_{i}^{\alpha}(t)\frac{\partial X^{i}_{a}}{\partial x^{j}}(x(t))u^{a}_{\alpha}(t)$. The extremum of the linear function $u\to H$ exists since the set $U$ is compact; for optimum, the control vectors $u_{\alpha}=(u^{1}_{\alpha},...,u^{n-1}_{\alpha})$ must be vertices of $\partial U$. If $Q^{\alpha}_{a}(t)=p^{\alpha}_{i}(t)X^{i}_{a}(x(t))$ are the switching functions, then each optimal control $u^{*a}_{\alpha}$ is of the form

[TABLE]

Suppose the Lebesgue measure of each set $\{t\in\Omega_{0\tau}:Q^{\alpha}_{a}(t)=0\}$ vanishes. Then the singular control is ruled out and the remaining possibilities are bang-bang controls. This optimal control is discontinuous since each component jumps from a minimum to a maximum and vice versa in response to each change in the sign of each $Q^{\alpha}_{a}(t)$. The piecewise complete integrability identities keep only the control vectors (vertices of $\partial U$) $u_{\alpha}$ which satisfy $u_{\alpha}=\pm\,u_{1}$. Each optimal $m$-sheet $x(t)$ is a soliton solution.

Case of curvilinear integral functional

Optimal terminal value Let $U=[-1,1]^{m(n-1)}\subset R^{m(n-1)}$ be the control set. Suppose we have to

Minimize the terminal cost functional

[TABLE]

subject to the driftless control system

[TABLE]

Solution Since the control Hamiltonian $H(x,p,u)=p_{i}X^{i}_{a}(x)u^{a}$ is linear in the control, the optimal control is a bang-bang. Automatically we find the optimal costate function and the optimal evolution.

6 Optimal control problems on

Tzitzeica surfaces

Let $R^{2}_{+}$ be endowed with the product order. Let $\Omega\subset R^{2}_{+}$ be the bi-dimensional interval determined by the opposite diagonal points $(0,0)$ and $(u^{1},v^{1})$.

Problem: find

[TABLE]

constrained by (non-ruled Tzitzeica surfaces)

[TABLE]

[TABLE]

where $h(u,v)$ is the control.

To solve the problem we use

[TABLE]

[TABLE]

Explicitely, find

[TABLE]

7 Phytoplankton growth model

Open problem Transform the next ODE systems into Pfaff systems and study their stochastic perturbations.

Alessandro Abate, Ashish Tiwari, Shankar Sastry, Box Invariance for biologically-inspired dynamical systems

(i) O. Bernard and J.-L. Gouze, ”Global qualitative description of a class of nonlinear dynamical systems,” Artificial Intelligence, vol. 136, pp. 29-59, 2002:

Consider the following Phytoplankton Growth Model:

[TABLE]

where $x^{1}$ denotes the substrate, $x^{2}$ the phytoplankton biomass, and $x^{3}$ the intracellular nutrient per biomass.

(ii) A. Julius, A. Halasz, V. Kumar, and G. Pappas, Controlling biological systems: the lactose regulation system of Escherichia Coli, in American Control Conference 2007:

The dynamics of tetracycline antibiotic in a bacteria which develops resistance to this drug (by turning on genes $TetA$ and $TetR$) can be described by the following hybrid system:

[TABLE]

[TABLE]

where $f=\frac{1}{2000}$ if $TetR>\frac{1}{50000}$ and $f=\frac{1}{40}$ otherwise ($f$ is the transcription rate of genes, which are inhibited by $TetR$), and $x^{1}$, $x^{2}$, $x^{3}$, $x^{4}$ are the cytoplasmic concentrations of $TetR$ protein, the $TetR-Tc$ complex, Tetracycline, and $TetA$ protein, and $u_{0}$ is the extracellular concentration of Tetracycline.

Acknowledgments

Partially supported by University Politehnica of Bucharest, by UNESCO Chair in Geodynamics, ”Sabba S. Ştefănescu” Institute of Geodynamics, Romanian Academy and by Academy of Romanian Scientists.