Dynamic Virtual Holonomic Constraints for Stabilization of Closed Orbits in Underactuated Mechanical Systems

TL;DR

This paper presents a novel control method for enforcing virtual holonomic constraints and stabilizing closed orbits in underactuated mechanical systems, demonstrated on a PVTOL aircraft to follow a circular path without rollover.

Contribution

It introduces a new approach using a double-integrator parameterization of VHCs to achieve both constraint enforcement and orbit stabilization in underactuated systems.

Findings

Successfully stabilizes closed orbits on the constraint manifold.

Ensures the aircraft follows a circular path with a specified speed.

Prevents rollover for suitable initial conditions.

Abstract

This article investigates the problem of enforcing a virtual holonomic constraint (VHC) on a mechanical system with degree of underactuation one while simultaneously stabilizing a closed orbit on the constraint manifold. This problem, which to date is open, arises when designing controllers to induce complex repetitive motions in robots. In this paper, we propose a solution which relies on the parameterization of the VHC by the output of a double-integrator. While the original control inputs are used to enforce the VHC, the control input of the double-integrator is designed to asymptotically stabilize the closed orbit and make the state of the double-integrator converge to zero. The proposed design is applied to the problem of making a PVTOL aircraft follow a circle on the vertical plane with a desired speed profile, while guaranteeing that the aircraft does not roll over for suitable initial conditions.

Equations

$$D(q)\ddot{q}+C(q,\dot{q})\dot{q}+\nabla P(q)=B(q)\tau ,$$

$$\Gamma :=\{(q,\dot{q}):h(q)=0,d{h}_q\dot{q}=0\},$$

$$T_q{h}^{-1}(0)\oplus \mathrm{Im}(D^{-1}(q)B(q))=T_q\mathcal{Q}.$$

$$(\forall \theta \in \mathbb{R})B^{\perp }(\sigma (\theta ))D(\sigma (\theta )){\sigma }^{\prime }(\theta )\ne 0.$$

$${\alpha }_1(\Vert (q,\dot{q}){\Vert }_{\Gamma })\le H(q,\dot{q})\le {\alpha }_2(\Vert (q,\dot{q}){\Vert }_{\Gamma }).$$

$$\tau =A^{-1}(q)\left\{d{h}_q{D}^{-1}(q)[C(q,\dot{q})\dot{q}+\nabla P(q)]-\mathcal{H}(q,\dot{q})-k_{p}e-k_d\dot{e}\right\},$$

$$\ddot{\theta }={\Psi }_1(\theta )+{\Psi }_2(\theta ){\dot{\theta }}^2,$$

$$\displaystyle\Psi_{1}(\theta)=-\frac{B^{\perp}\nabla P}{B^{\perp}D\sigma^{\prime}}\bigg{|}_{q=\sigma(\theta)},$$

$$\displaystyle\Psi_{2}(\theta)=-\frac{B^{\perp}D\sigma^{{\prime\prime}}+\sum_{i=1}^{n}B^{\perp}_{i}\sigma^{\prime^{\top}}Q_{i}\sigma^{\prime}}{B^{\perp}D\sigma^{\prime}}\Bigg{|}_{q=\sigma(\theta)},$$

$$M(\theta ):=\exp \left(-2{\int }_0^{\theta }{\Psi }_2(\tau )d\tau \right),V(\theta ):=-{\int }_0^{\theta }{\Psi }_1(\tau )M(\tau )d\tau .$$

$$E(\theta ,\dot{\theta })=\frac{1}{2}M(\theta ){\dot{\theta }}^2+V(\theta ).$$

$$\dot{\theta }=\pm \sqrt{\frac{2}{M(\theta )}(E_0-V(\theta ))},$$

$$\vartheta \mapsto ({\phi }_1(\vartheta ),{\phi }_2(\vartheta ))=\left(\vartheta ,\pm \sqrt{\frac{2}{M(\vartheta )}(E_0-V(\vartheta ))}\right).$$

$$(\theta ,\dot{\theta })\mapsto \left(\theta ,T(\theta )\dot{\theta }\right),T(\theta )=\sqrt{\frac{R^2-(\theta -C{)}^2}{\frac{2}{M(\theta )}(E_0-V(\theta ))}}.$$

$${\theta }^1:=\underset{(\theta ,\dot{\theta })\in \gamma }{\min }\theta ,{\theta }^2:=\underset{(\theta ,\dot{\theta })\in \gamma }{\max }\theta ,C:=({\theta }^1+{\theta }^2)/2,R:=({\theta }^2-{\theta }^1)/2.$$

$$\vartheta \mapsto ({\phi }_1(\vartheta ),{\phi }_2(\vartheta ))=\left(C+R\cos (\vartheta ),\frac{R\sin (\vartheta )}{T(C+R\cos (\vartheta ))}\right).$$

$$\bar{\Gamma }=\{(q,\dot{q},s,\dot{s}):h^s(q)=0,{\partial }_q{h}^s\dot{q}+{\partial }_s{h}^s\dot{s}=0\}.$$

$$\Vert (q,\dot{q},s,\dot{s}){\Vert }_{\bar{\Gamma }}=\Vert (q-Ls,\dot{q}-L\dot{s}){\Vert }_{\Gamma },$$

$$\Vert (q,\dot{q}){\Vert }_{\Gamma }\le \Vert (q,\dot{q},s,\dot{s}){\Vert }_{\bar{\Gamma }}+\Vert (Ls,L\dot{s})\Vert .$$

$$\bar{\gamma }:=\{(q,\dot{q},s,\dot{s}):s=\dot{s}=0,q=\sigma (\theta ),\dot{q}={\sigma }^{\prime }(\theta )\dot{\theta },(\theta ,\dot{\theta })\in [\mathbb{R}{]}_{T_1}\times \mathbb{R},E(\theta ,\dot{\theta })=E_0\}.$$

$$D(q)\ddot{q}+C(q,\dot{q})\dot{q}+\nabla P(q)$$

$$\ddot{s}$$

$$\bar{\Gamma }:=\{(q,\dot{q},s,\dot{s}):h^s(q)=0,{\partial }_q{h}^s\dot{q}+{\partial }_s{h}^s\dot{s}=0\},$$

$$\ddot{e}=(\star )+A^s(q)\tau +B^s(q)v,$$

$$h^s(q):=h(q-Ls),$$

$$\bar{\Gamma }$$

$$=\{(q,\dot{q},s,\dot{s}):(q-Ls,\dot{q}-L\dot{s})\in \Gamma \}.$$

$$\bar{\gamma }=\{(q,\dot{q},s,\dot{s}):s=\dot{s}=0,q=\sigma (\theta ),\dot{q}={\sigma }^{\prime }(\theta )\dot{\theta },(\theta ,\dot{\theta })\in [\mathbb{R}{]}_{T_1}\times \mathbb{R},E(\theta ,\dot{\theta })=E_0\}.$$

$$\tau^{\star}(q,\dot{q},s,\dot{s},v)=\big{(}A^{s}(q)\big{)}^{-1}\Big{\{}dh_{q-Ls}\,D^{-1}(q)\big{[}C(q,\dot{q})\dot{q}+\nabla P(q)\big{]}+dh_{q-Ls}\,Lv-\mathcal{H}(q,\dot{q},s,\dot{s})-k_{p}e-k_{d}\dot{e}\Big{\}},$$

$$q=\sigma (\theta )+Ls,\dot{q}={\sigma }^{\prime }(\theta )\dot{\theta }+L\dot{s},\ddot{q}={\sigma }^{\prime }(\theta )\ddot{\theta }+{\sigma }^{\prime \prime }(\theta ){\dot{\theta }}^2+Lv.$$

$$\ddot{\theta }={\Psi }_1^s(\theta )+{\Psi }_2^s(\theta ){\dot{\theta }}^2+{\Psi }_3^s(\theta )\dot{\theta }\dot{s}+{\Psi }_4^s(\theta ){\dot{s}}^2+{\Psi }_5^s(\theta )v,$$

$$\ddot{s}=v,$$

Full text

††thanks: This paper was not presented at any IFAC meeting. Corresponding author M. Maggiore. Tel. (Fax) +1-416-946-5095.

, ,

Virtual holonomic constraints (VHCs) have been recognized to be key to solving complex motion control problems in robotics. There is an increasing body of evidence from bipedal robotics [12, 13, 34], snake robot locomotion [27], and repetitive motion planning [1, 31] that VHCs constitute a new motion control paradigm, an alternative to the traditional reference tracking framework. The key difference with the standard motion control paradigm of robotics is that, in the VHC framework, the desired motion is parameterized by the states of the mechanical system, rather than by time.

Geometrically, a VHC is a subset111More precisely, an embedded submanifold. of the configuration manifold of the mechanical system. Enforcing a VHC means stabilizing the subset of the state space of the mechanical system where the generalized coordinates of the mechanical system satisfy the VHC, while the generalized velocity is tangent to the VHC. This subset is called the constraint manifold.

Grizzle and collaborators (see, e.g., [34]) have shown that the enforcement of certain VHCs on a biped robot leads, under certain conditions, to the orbital stabilization of a hybrid closed orbit corresponding to a repetitive walking gait. The orbit in question lies on the constraint manifold, and the mechanism stabilizing it is the dissipation of energy that occurs when a foot impacts the ground. In a mechanical system without impacts, this stabilization mechanism disappears, and the enforcement of the VHC alone is insufficient to achieve the ultimate objective of stabilizing a repetitive motion. Some researchers [32, 30] have addressed this problem by using the VHC exclusively for motion planning, i.e., to find a desired closed orbit. Once a suitable closed orbit is found, a time-varying controller is designed by linearizing the control system along the orbit. In this approach, the constraint manifold is not an invariant set for the closed-loop system, and thus the VHC is not enforced via feedback.

To the best of our knowledge, for mechanical control systems with degree of underactuation one, the problem of simultaneous enforcement of a VHC and orbital stabilization of a closed orbit lying on the constraint manifold is still open. The challenge in addressing this problem lies in the fact that the dynamics of the mechanical control system on the constraint manifold are unforced. Therefore, any feedback that asymptotically stabilizes the desired closed orbit cannot render the constraint manifold invariant, and thus cannot enforce the VHC. To overcome this difficulty, in this paper we propose to render the VHC dynamic. By doing that, under suitable assumptions it is possible to stabilize the desired closed orbit while simultaneously enforcing the dynamic VHC.

Contributions of the paper. This paper presents the first solution of the simultaneous stabilization problem just described for mechanical control systems with degree of underactuation one. Leveraging recent results in [26], we consider VHCs that induce Lagrangian constrained dynamics. The closed orbits on the constraint manifold are level sets of a “virtual” energy function. We make the VHC dynamic by parametrizing it by the output of a double-integrator. We use the original control inputs of the mechanical system to stabilize the constraint manifold associated with the dynamic VHC, and we use the double-integrator input to asymptotically stabilize the selected orbit on the constraint manifold. Because the output of the double-integrator acts as a perturbation of the original constraint manifold, we also make sure that the state of the double-integrator converges to zero. To achieve these objectives, we develop a novel theoretical result giving necessary and sufficient conditions for the exponential stabilizability of closed orbits for control-affine systems.

The benefits associated with the simultaneous stabilization proposed in this paper are as follows. First, in the proposed framework one may assign the speed of convergence of solutions to the constraint manifold independently of the orbit stabilization mechanism. In particular, one may enforce the dynamic VHC arbitrarily fast222Naturally, actuator saturation will limit the maximum attainable speed of convergence to the constraint manifold., so that after a short transient, the qualitative behaviour of trajectories of the closed-loop system is determined by the dynamic VHC. Second, since the constraint manifold is asymptotically stable for the closed-loop system, trajectories originating near the constraint manifold remain close to it thereafter. From a practical standpoint, the two features just highlighted mean that the dynamic VHC offers some control over the transient behaviour of the closed-loop system. The simultaneous stabilization of the closed orbit means that, without violating the dynamic VHC, an extra stabilization mechanism makes the trajectories of the closed-loop system converge to the closed orbit.

The property just described is illustrated in this paper with an example, the model of a PVTOL aircraft moving along a unit circle on the vertical plane. The control specification is to make the aircraft traverse the circle with bounded speed, while guaranteeing that the aircraft does not undergo full revolutions along its longitudinal axis. In this context, the VHC constrains the roll angle of the aircraft as a function of its position on the circle, preventing the aircraft from rolling over. On the other hand, the simultaneous stabilization of the closed orbit corresponds to stabilizing a desired periodic speed profile on the circle without violating the constraint. The double-integrator state perturbs the constraint so as to induce the orbit stabilization mechanism.

Relevant literature. Previous work employs VHCs to stabilize desired closed orbits for underactuated mechanical systems [5, 4, 32, 10]. Canudas-de-Wit and collaborators [5] propose a technique to stabilize a desired closed orbit that relies on enforcing a virtual constraint and on dynamically changing its geometry so as to impose that the reduced dynamics on the constraint manifold match the dynamics of a nonlinear oscillator. In [4, 32], Canudas-de-Wit, Shiriaev, and collaborators employ VHCs to aid the selection of closed orbits of underactuated mechanical systems. It is demonstrated that an unforced second-order system possessing an integral of motion describes the constrained motion. Assuming that this unforced system has a closed orbit, a linear time-varying controller is designed that yields exponential stability of the closed orbit. With the exception of [5], the papers above do not guarantee the invariance of the VHC for the closed loop system. The idea of event-triggered dynamic VHCs has appeared in the work by Morris and Grizzle in [28] where the authors construct a hybrid invariant manifold for the closed-loop dynamics of biped robots by updating the VHC parameters after each impact with the ground. This approach is similar in spirit to the one presented in this paper. Finally, the paper [6] discusses collocated VHCs, i.e., VHCs parametrized by actuated variables. In Section 6, we discuss the differences between the method presented in this article and the ones in [5, 4, 32, 30]. We also discuss the conceptual similarities between the method presented in this article and the one in [28].

Organization. This article is organized as follows. We review preliminaries in Section 1. The formal problem statement and our solution strategy are presented in Section 2. In Section 3 we present dynamic VHCs. In Section 4 we present a novel result of a general nature providing necessary and sufficient conditions for the exponential stabilizability of closed orbits for control-affine systems, and use it to design the input of the double-integrator to stabilize the closed orbit relative to the constraint manifold. In Section 5 we present the complete control law solving the VHC-based orbital stabilization problem. In Section 6 we discuss the differences between the method presented in this article and the ones in [4, 5, 32]. Finally, in Section 7 we apply the ideas of this paper to a path following problem for the PVTOL aircraft.

Notation. If $x\in\mathbb{R}$ and $T>0$, then $x$ modulo $T$ is denoted by $[x]_{T}$, and the set $\{[x]_{T}:x\in\mathbb{R}\}$ is denoted by $[\mathbb{R}]_{T}$. This set can be given a manifold structure which makes it diffeomorphic to the unit circle $\mathbb{S}^{1}$. If $a$ and $b$ are vectors, then $\operatorname{col}(a,b):=[a^{\top}\ b^{\top}]^{\top}$. If $a,b\in\mathbb{R}^{n}$, we denote $\langle a,b\rangle=a^{\top}b$, and $\|a\|=\langle a,a\rangle^{1/2}$. If $A\in\mathbb{R}^{n\times n}$, we denote by $\|A\|_{2}$ the induced two-norm of $A$. If $(\mathcal{X},d)$ is a metric space, $\Gamma$ is a subset of $\mathcal{X}$, and $x\in\mathcal{X}$, we denote by $\|x\|_{\Gamma}$ the point-to-set distance of $x$ to $\Gamma$, defined as $\|x\|_{\Gamma}:=\inf_{y\in\Gamma}d(x,y)$.

If $h:M\to N$ is a smooth map between smooth manifolds, and $q\in M$, we denote by $dh_{q}:T_{q}M\to T_{h(q)}N$ the derivative of $h$ at $q$ (in coordinates, this is the Jacobian matrix of $h$ evaluated at $q$), and if $M$ has dimension 1, then we may use the notation $h^{\prime}(q)$ in place of $dh_{q}$. If $M_{1},M_{2},N$ are smooth manifolds and $f:M_{1}\times M_{2}\to N$ is a smooth function, then $\partial_{q_{1}}f(q_{1},q_{2})$ denotes the derivative of the map $q_{1}\mapsto f(q_{1},q_{2})$ at $q_{1}$. If $f:M\to TM$ is a vector field on $M$ and $h:M\to\mathbb{R}^{m}$ is $C^{1}$, then $L_{f}h:M\to\mathbb{R}^{m}$ is defined as $L_{f}h(q):=dh_{q}f(q)$. For a function $h:M\to\mathbb{R}^{m}$, we denote by $h^{-1}(0):=\{q\in M:h(q)=0\}$.

If $A\in\mathbb{R}^{m\times n}$ has full row-rank, we denote by $A^{\dagger}$ the pseudoinverse of $A$, $A^{\dagger}=A^{\top}(AA^{\top})^{-1}$. Given a $C^{2}$ scalar function $f:\mathbb{R}^{n}\to\mathbb{R}$, we denote by $\mbox{Hess}(f)$ its Hessian matrix.

1 Preliminaries

Consider the underactuated mechanical control system

[TABLE]

where q=\big{(}q_{1},\ldots,q_{n}\big{)}\in\mathcal{Q} is the configuration vector with $q_{i}$ either a displacement in $\mathbb{R}$ or an angular variable in $[\mathbb{R}]_{T_{i}}$, with $T_{i}>0$. The configuration space $\mathcal{Q}$ is, therefore, a generalized cylinder. In (1), $B:\mathcal{Q}\to\mathbb{R}^{n\times n-1}$ is $C^{1}$ and it has full rank $n-1$. Also, $D(q)$, the inertia matrix, is positive definite for all $q$, and $P(q)$, the potential energy function, is $C^{1}$. We assume that there exists a left-annihilator of $B(q)$; specifically, there is a $C^{1}$ function $B^{\perp}:\mathcal{Q}\to\mathbb{R}^{1\times n}\backslash\{0\}$ such that $B^{\perp}(q)B(q)=0$ for all $q\in\mathcal{Q}$.

Definition 1.1** ([25]).**

A relation $h(q)=0$, where $h:\mathcal{Q}\to\mathbb{R}^{k}$ is $C^{2}$, is a regular virtual holonomic constraint (VHC) of order $k$ for system (1), if (1) with output $e=h(q)$ has well-defined vector relative degree $\{2,\cdots,2\}$ everywhere on the constraint manifold

[TABLE]

i.e., the matrix $dh_{q}D^{-1}(q)B(q)$ has full row rank for all $q\in h^{-1}(0)$.

The constraint manifold $\Gamma$ in (2) is just the zero dynamics manifold associated with the output $e=h(q)$. For a VHC of order $n-1$, the set $h^{-1}(0)$ is a collection of disconnected regular curves, each one diffeomorphic to either the unit circle or the real line. From now on, we will assume that $h^{-1}(0)$ is diffeomorphic to $\mathbb{S}^{1}$.

Necessary and sufficient conditions for a relation $h(q)=0$ to be a regular VHC of order $n-1$ are given in the following proposition.

Proposition 1** ([25]).**

Let $h:\mathcal{Q}\to\mathbb{R}^{n-1}$ be $C^{2}$ and such that $\operatorname{rank}dh_{q}=n-1$ for all $q\in h^{-1}(0)$. Then, $h(q)=0$ is a regular VHC of order $n-1$ for system (1) if and only if for each $q\in h^{-1}(0)$,

[TABLE]

Moreover, if $\sigma:\mathbb{R}\rightarrow\mathcal{Q}$ is a regular parameterization of $h^{-1}(0)$, then $h(q)=0$ is a regular VHC for system (1) if and only if

[TABLE]

By definition, if $h:\mathcal{Q}\to\mathbb{R}^{n-1}$ is a regular VHC, system (1) with output $e=h(q)$ has vector relative degree $\{2,\ldots,2\}$. In order to asymptotically stabilize the constraint manifold, one may employ an input-output linearizing feedback, as detailed in the next proposition. Before stating the proposition, we define the notion of set stability used in this paper.

Definition 1.2**.**

Consider a dynamical system $\Sigma$ on a metric space $(\mathcal{X},d)$ with continuous local flow map $\phi(t,x_{0})$, defined on an open subset of $\mathbb{R}\times\mathcal{X}$. The set $\Gamma\subset\mathcal{X}$ is stable for $\Sigma$ if for all $\varepsilon>0$ there exists a neighbourhood $U$ of $\Gamma$ such that for all $x_{0}\in U$ such that $\phi(t,x_{0})$ is defined for all $t\geq 0$, $\|\phi(t,x_{0})\|_{\Gamma}<\varepsilon$ for all $t\geq 0$. The set $\Gamma$ is asymptotically stable for $\Sigma$ if it is stable and there exists a neighbourhood $U$ of $\Gamma$ such that for all $x_{0}\in U$ such that $\phi(t,x_{0})$ is defined for all $t\geq 0$, $\|\phi(t,x_{0})\|_{\Gamma}\to 0$ as $t\to\infty$.

We remark that if $\gamma\subset\mathcal{X}$ is a closed orbit of the dynamical system $\Sigma$, then the notion of asymptotic stability of $\gamma$ coincides with that of asymptotic orbital stability found in the literature (see, e.g., [22, Definition 8.2]). Therefore, in the sequel we will speak of asymptotic stability of a closed orbit $\gamma$.

Proposition 2** ([25]).**

Let $h(q)=0$ be a regular VHC of order $n-1$ for system (1) with associated constraint manifold $\Gamma$ in (2). Let $H(q,\dot{q})=\operatorname{col}(h(q),dh_{q}\dot{q})$, and assume that there exist two class-$\mathcal{K}$ functions $\alpha_{1},\alpha_{2}$ such that

[TABLE]

Let $A(q)=dh_{q}D^{-1}(q)B(q)$, $e=h(q)$, and $\mathcal{H}=\operatorname{col}(\mathcal{H}_{1},\ldots,\mathcal{H}_{n-1})$, where $\mathcal{H}_{i}=\dot{q}^{\top}\operatorname{Hess}(h_{i})\dot{q}$. Then, for all $k_{p},k_{d}>0$, the input-output linearizing controller

[TABLE]

asymptotically stabilizes the constraint manifold $\Gamma$.

Once the constraint manifold $\Gamma$ has been rendered invariant by the above feedback, the motion on $\Gamma$ is described by a second-order unforced differential equation, as detailed in the next proposition.

Proposition 3** ([35, 32, 26]).**

Let $h(q)=0$ be a regular VHC of order $n-1$ for system (1). Assume that $h^{-1}(0)$ is diffeomorphic to $\mathbb{S}^{1}$. For some ${T_{1}}>0$, let $\sigma:[\mathbb{R}]_{{T_{1}}}\rightarrow\mathcal{Q}$ be a regular parameterization of $h^{-1}(0)$. Then letting $(q,\dot{q})=(\sigma(\theta),\sigma^{\prime}(\theta)\dot{\theta})$, the dynamics on the set $\Gamma$ in (2) are globally described by

[TABLE]

where $(\theta,\dot{\theta})\in[\mathbb{R}]_{T_{1}}\times\mathbb{R}$ and

[TABLE]

and where $B^{\perp}_{i}$ is the $i$-th component of $B^{\perp}$ and $Q_{i}$ is an $n\times n$ matrix whose $(j,k)$-th component is $(Q_{i})_{jk}=(1/2)(\partial_{q_{k}}D_{ij}+\partial_{q_{j}}D_{ik}-\partial_{q_{i}}D_{kj})$.

Henceforth, we will refer to (4) as the reduced dynamics. System (4) is unforced since all $n-1$ control directions are used to make the constraint manifold $\Gamma$ invariant. In the context of nonlinear control, the reduced dynamics (4) are a coordinate representation of the zero dynamics vector field of the mechanical system (1) with output $e=h(q)$. Proposition 3 is a direct consequence of the fact that outputs of a nonlinear systems with a well-defined relative degree induce a globally-defined zero dynamics vector field [20].

Remark 4**.**

The proposition above states that the second-order differential equation (4) with state space $[\mathbb{R}]_{{T_{1}}}\times\mathbb{R}$ represents the dynamics on the set $\Gamma$. The geometric underpinning of this statement is the fact that $\Gamma$ is diffeomorphic to the cylinder $[\mathbb{R}]_{{T_{1}}}\times\mathbb{R}$ via the diffeomorphism $[\mathbb{R}]_{{T_{1}}}\times\mathbb{R}\to\Gamma$, $(\theta,\dot{\theta})\mapsto(\sigma(\theta),\sigma^{\prime}(\theta)\dot{\theta})$. We can therefore identify $\Gamma$ with the cylinder $[\mathbb{R}]_{{T_{1}}}\times\mathbb{R}$, and parametrize it with the variables $(\theta,\dot{\theta})$, see Figure 1. $\triangle$

Under certain conditions, the reduced dynamics (4) have a Lagrangian structure. Define

[TABLE]

Proposition 5** ( [26]).**

Consider the reduced dynamics (4) with state space $[\mathbb{R}]_{{T_{1}}}\times\mathbb{R}$. System (4) is Lagrangian if and only if the functions $M(\cdot)$ and $V(\cdot)$ in (6) are ${T_{1}}$-periodic, in which case the Lagrangian function is given by $L(\theta,\dot{\theta})=(1/2)M(\theta)\dot{\theta}^{2}-V(\theta)$.

An immediate consequence of the foregoing result is that, when the reduced dynamics (4) are Lagrangian, the orbits of (4) are characterized by the level sets of the energy function

[TABLE]

We remark that the energy function $E(\theta,\dot{\theta})$ appeared in the work [11]. A different function, dependent on initial conditions, was presented in [32] as an “integral of motion” of the reduced dynamics (4).

Almost all orbits of the reduced dynamics (4) are closed, and they belong to two distinct families, defined next.

Definition 1.3**.**

A closed orbit $\gamma$ of the reduced dynamics (4) is said to be a rotation of $\theta$ if $\gamma$ is homeomorphic to a circle $\{(\theta,\dot{\theta})\in[\mathbb{R}]_{T}\times\mathbb{R}:\dot{\theta}=\text{constant}\}$ via a homeomorphism of the form $(\theta,\dot{\theta})\mapsto(\theta,T(\theta)\dot{\theta})$; $\gamma$ is an oscillation of $\theta$ if it is homeomorphic to a circle $\{(\theta,\dot{\theta})\in[\mathbb{R}]_{{T_{1}}}\times\mathbb{R}:\theta^{2}+\dot{\theta}^{2}=\text{constant}\}$ via a homeomorphism of the form $(\theta,\dot{\theta})\mapsto(\theta,T(\theta)\dot{\theta})$.

In [25, Proposition 4.7], it is shown that if the assumptions of Proposition 3 hold, then almost all orbits of (4) are either oscillations or rotations. Oscillations and rotations are illustrated in Figure 2. It is possible to give an explicit regular parameterization of rotations and oscillations which will be useful in what follows.

If $\gamma$ is a rotation with associated energy value $E_{0}$, then we may solve $E(\theta,\dot{\theta})=E_{0}$ for $\dot{\theta}$ obtaining

[TABLE]

with plus sign for counterclockwise rotation, and minus sign for clockwise rotation. Thus a rotation $\gamma$ is the graph of a function, which leads to the natural regular parameterization $[\mathbb{R}]_{{T_{1}}}\to[\mathbb{R}]_{{T_{1}}\times\mathbb{R}}$ given by

[TABLE]

Concerning oscillations, it was shown in [7, Lemma 3.12] that they are mapped homeomorphically to circles via the homeomorphism

[TABLE]

In the above, $E_{0}$ is the energy level associated with $\gamma$, and

[TABLE]

The image of $\gamma$ under the above homeomorphism is a circle of radius $R$ centred at $(C,0)$. Using this fact, we get the following regular parameterization $[\mathbb{R}]_{2\pi}\to[\mathbb{R}]_{{T_{1}}}\times\mathbb{R}$:

[TABLE]

2 Problem formulation

Consider the mechanical control system (1) with $n$ DOFs and $n-1$ control inputs. Let $h(q)=0$ be a regular VHC of order $n-1$, and assume that $h^{-1}(0)$ is diffeomorphic to $\mathbb{S}^{1}$. As before, let $\sigma:[\mathbb{R}]_{{T_{1}}}\to\mathcal{Q}$, ${T_{1}}>0$, be a regular parameterization of $h^{-1}(0)$.

Assume that the dynamics (4) are Lagrangian, so that almost all of its closed orbits are rotations or oscillations. In particular, almost every orbit of the reduced dynamics on the constraint manifold is closed, and it corresponds to a certain speed profile. Pick one such orbit of interest333Here we assume that the level set $\{(\theta,\dot{\theta}):E(\theta,\dot{\theta})=E_{0}\}$ is connected. There is no loss of generality in this assumption, since the theory developed below relies on the regular parametrizations (8), (9), which one can use to select one of the desired connected components of $\gamma$., $\gamma=\{(\theta,\dot{\theta}):E(\theta,\dot{\theta})=E_{0}\}$. As pointed out in the introduction, since the reduced dynamics (4) are unforced, it is impossible to stabilize this orbit while preserving the invariance of the constraint manifold.

The idea we explore in this paper is to introduce a dynamic perturbation of the constraint manifold. We define a one-parameter family of VHCs $h^{s}(q)=0$, where $s\in\mathbb{R}$ is the parameter and the map $h^{s}(q)$ is such that $h^{0}(q)=h(q)$. In Section 3 of this paper we choose the parametrization $h^{s}(q):=h(q-Ls)$, where $L\in\mathbb{R}^{n}$ is a parameter vector, but other parametrizations are possible. Each VHC in the one-parameter family can be viewed as a perturbation of the original VHC $h(q)=0$. The parameter $s$ is dynamically adapted by means of the double-integrator $\ddot{s}=v$, where $v$ is a new control input. System (1), augmented with this double-integrator, has state $(q,\dot{q},s,\dot{s})\in\mathcal{Q}\times\mathbb{R}^{n}\times\mathbb{R}\times\mathbb{R}$, and control input $(\tau,v)\in\mathbb{R}^{n-1}\times\mathbb{R}$. We will employ $\tau$ to stabilize the constraint manifold associated with the dynamic VHC $h^{s}(q)=0$, and $v$ to stabilize a new closed orbit for the augmented system. We will detail our solution steps below, but first we will formulate precisely our control specifications.

We begin by defining the constraint manifold associated with the family of VHCs $h^{s}(q)=0$ as

[TABLE]

The original constraint manifold $\Gamma$ is embedded in $\bar{\Gamma}$ as the intersection of $\bar{\Gamma}$ with the plane \big{\{}(q,\dot{q},s,\dot{s}):\,s=0,\,\dot{s}=0\big{\}}, because the identity $h^{0}(q)=h(q)$ implies that $\{(q,\dot{q},s,\dot{s}):(q,\dot{q})\in\Gamma,(s,\dot{s})=(0,0)\}\subset\bar{\Gamma}$. For the one-parameter family $h^{s}(q)=h(q-Ls)$ used in this paper, we will show in the proof of Proposition 7 that the distance of a point $(q,\dot{q},s,\dot{s})\in\mathcal{Q}\times\mathbb{R}^{n}\times\mathbb{R}\times\mathbb{R}$ to the set $\bar{\Gamma}$ can be expressed as

[TABLE]

from which one readily deduces the inequality

[TABLE]

Thus, if the state $(q,\dot{q},s,\dot{s})$ of the augmented system is close to $\bar{\Gamma}$, and if $\|(s,\dot{s})\|$ is small, the state $(q,\dot{q})$ of the mechanical system is close to $\Gamma$, the original constraint manifold. For this reason, our first control specification is the asymptotic stabilization of $\bar{\Gamma}$.

The second control specification for the augmented system will correspond, in an appropriate manner, to the orbital stabilization of $\gamma$. The curve $\gamma$ is contained in the state space of the original mechanical system, so we need to lift it to the state space of the augmented system. The lift in question is

[TABLE]

The second control specification is the asymptotic stabilization of $\bar{\gamma}$. That this is indeed the right control specification follows from the observation that if $(q,\dot{q},s,\dot{s})\in\bar{\gamma}$, then $(q,\dot{q})\in\gamma$. Thus, the stabilization of $\bar{\gamma}$ for the augmented system makes the trajectories of the mechanical system (1) converge to $\gamma$, as desired. Moreover, when trajectories are close to $\bar{\gamma}$, $\|(s,\dot{s})\|$ is small, implying that $(q,\dot{q})$ is close to the original constraint manifold $\Gamma$, as argued above.

The two control specifications we have defined so far, namely the asymptotic stabilization of both $\bar{\Gamma}$ and $\bar{\gamma}$, are somewhat related to one another in that $\bar{\gamma}\subset\bar{\Gamma}$. Indeed, on $\bar{\gamma}$ one has that $(q,\dot{q})=(\sigma(\theta),\sigma^{\prime}(\theta)\dot{\theta})\in\Gamma$ and $(s,\dot{s})=(0,0)$, which readily implies that $(q,\dot{q},s,\dot{s})\in\bar{\Gamma}$.

In summary, we have formulated the following

VHC-based orbital stabilization problem. Find a smooth control law for system (1) augmented with the double-integrator $\ddot{s}=v$ that asymptotically stabilizes both sets $\bar{\gamma}\subset\bar{\Gamma}$.

We recall from the foregoing discussion that the asymptotic stabilization of $\bar{\Gamma}$ corresponds to the enforcement of the perturbed VHC $h^{s}(q)=0$. Since $(s,\dot{s})=(0,0)$ on $\bar{\gamma}$, near $\bar{\gamma}$ the Hausdorff distance444The Hausdorff distance between two sets measures how far the two sets are from each other. between the set $\bar{\Gamma}$ and the original constraint manifold $\Gamma\times\{(s,\dot{s})=(0,0)\}$ is small. Considering the fact that $h(q)=0$ embodies a useful constraint that we wish to hold during the transient, the philosophy of the VHC-based orbital stabilization problem is to preserve as much as possible the beneficial properties of the original VHC $h(q)=0$, while simultaneously stabilizing the closed orbit $\gamma$ corresponding to a desired repetitive motion.

Solution steps. Our solution to the VHC-based orbital stabilization problem unfolds in three steps:

We present a technique to parameterize the VHC $h(q)=0$ with the output of a double integrator, giving rise to a dynamic VHC $h^{s}(q)=0$ with associated constraint manifold $\bar{\Gamma}$. We show that if the original VHC is regular, so too is its dynamic counterpart for small values of the double integrator output (Proposition 6). Moreover, if the original constraint manifold $\Gamma$ is stabilizable, so too is the perturbed manifold $\bar{\Gamma}$ (Proposition 7). We derive the reduced dynamics on this manifold, which are now affected by the input $v$ of the double integrator. 2. 2.

We develop a general result for control-affine systems (Theorem 8) relating the exponential stabilizability of a closed orbit to the controllability of a linear periodic system, for which we give an explicit representation. Leveraging this result, we design the input of the double-integrator, $v$, to exponentially stabilize the orbit relative to the manifold $\bar{\Gamma}$. 3. 3.

We put together the controller enforcing the dynamic VHC in Step 1 with the controller stabilizing the orbit in Step 2 and show that the resulting controller solves the VHC-based orbital stabilization problem (Theorem 12).

3 Step 1: Making the VHC dynamic

In this section we present the notion of dynamic VHCs. We begin by augmenting the dynamics in (1) with a double-integrator, to obtain the augmented system

[TABLE]

Henceforth, we use overbars to distinguish objects associated with the augmented control system (10) from those associated with (1). Accordingly, we define $\bar{q}:=(q,s)$, $\dot{\bar{q}}:=(\dot{q},\dot{s})$, \bar{\mathcal{Q}}:=\big{\{}(q,s):q\in\mathcal{Q},\,s\in\mathcal{I}\big{\}}.

Definition 3.1**.**

Let $h(q)=0$ be a regular VHC of order $n-1$ for system (1). A dynamic VHC based on $h(q)=0$ is a relation $h^{s}(q)=0$ such that the map $(s,q)\mapsto h^{s}(q)$ is $C^{2}$, $h^{0}(q)=h(q)$, and the parameter $s$ satisfies the differential equation $\ddot{s}=v$ in (10).

The dynamic VHC $h^{s}(q)$ is regular for (10) if there exists an open interval $\mathcal{I}\subset\mathbb{R}$ containing $s=0$ such that, for all $s\in\mathcal{I}$ and all $v\in\mathbb{R}$, system (10) with input $\tau$ and output $e=h^{s}(q)$ has vector relative degree $\{2,\cdots,2\}$.

The dynamic VHC $h^{s}(q)=0$ is stabilizable for (10) if there exists a smooth feedback $\tau(q,\dot{q},s,\dot{s},v)$ such that the manifold

[TABLE]

is asymptotically stable for the closed-loop system.

The reason for parameterizing the VHC with the output of a double integrator is to guarantee that the input $v$ of the double integrator appears after taking two derivatives of the output function $e=h^{s}(q)$. The regularity property of $h^{s}(q)$ in the foregoing definition means that, upon calculating the second derivative of $e=h^{s}(q)$ along the vector field in (10), the control input $\tau$ appears nonsingularly, i.e.,

[TABLE]

where $A^{s}$ and $B^{s}$ are suitable matrices, and $A^{s}$ is invertible for all $q\in(h^{s})^{-1}(0)$ and all $s\in\mathcal{I}$.

Given a regular VHC $h(q)=0$, a possible way to generate a dynamic VHC based on $h(q)=0$ is to translate the curve $h^{-1}(0)$ by an amount proportional to $s\in\mathbb{R}$. Other choices are of course possible, but this one has the benefit of allowing for simple expressions in the derivations that follow. We thus consider the following one-parameter family of mappings

[TABLE]

where $L\in\mathbb{R}^{n}$ is a non-zero constant vector. The zero level set of each family member in (12) is $(h^{s})^{-1}(0)=\{q+Ls:q\in h^{-1}(0)\}$, a translation555Recall that $q$ is a $n$-tuple whose $i$-th element, $q_{i}$, is either a real number or an element of $[\mathbb{R}]_{T_{i}}$. In the latter case, the sum $q_{i}+L_{i}s$ is to be understood as sum modulo $T_{i}$. of $h^{-1}(0)$ by the vector $Ls$ (see Figure 3). If $\sigma:[\mathbb{R}]_{{T_{1}}}\rightarrow\mathcal{Q}$ is a regular parameterization of the curve $h^{-1}(0)$, a regular parameterization of the zero level set of each family member in (12) is $\sigma^{s}(\theta)=\sigma(\theta)+Ls$. In an analogous manner, the constraint manifold $\bar{\Gamma}$ in (11) is the translation of $\Gamma$ by the vector $\operatorname{col}(Ls,\,L\dot{s})$,

[TABLE]

In the augmented coordinates, the closed orbit we wish to stabilize is

[TABLE]

The next two propositions show that if $h(q)=0$ is regular and stabilizable, so too is its dynamic counterpart $h(q-Ls)=0$. Their proofs are in Appendix A.

Proposition 6**.**

If $h(q)=0$ is a regular VHC of order $n-1$ for (1), then for any $L\in\mathbb{R}^{n}$ the dynamic VHC $h(q-Ls)=0$ is regular for the augmented system (10).

Proposition 7**.**

If $h(q)=0$ is a regular VHC of order $n-1$ for (1) satisfying the stabilizability condition (3), then for any $L\in\mathbb{R}^{n}$, the dynamic VHC $h(q-Ls)=0$ is stabilizable in the sense of Definition 3.1, and a feedback stabilizing the constraint manifold $\bar{\Gamma}$ in (13) is $\tau=\tau^{\star}(q,\dot{q},s,\dot{s},v)$ given by

[TABLE]

where $e=h(q-Ls)$, $\dot{e}=dh_{q-Ls}\,(\dot{q}-L\dot{s})$, $A^{s}(q)=dh_{q-Ls}\,D^{-1}(q)B(q)$, $k_{p}$, $k_{d}>0$, and $\mathcal{H}=\operatorname{col}(\mathcal{H}_{1},\ldots,\mathcal{H}_{n-1})$, $\mathcal{H}_{i}=(\dot{q}-L\dot{s}){{}^{\top}}\operatorname{Hess}(h_{i})|_{q-Ls}(\dot{q}-L\dot{s})$.

Next, we find the reduced dynamics of the augmented system (10) with feedback (15) on the manifold $\bar{\Gamma}$ in (13). To this end, we left-multiply (10) by the left annihilator $B^{\perp}$ of $B$ and evaluate the resulting equation on $\bar{\Gamma}$ by setting

[TABLE]

By so doing, one obtains:

[TABLE]

where

[TABLE]

The two second-order differential equations (16) will be henceforth referred to as the extended reduced dynamics induced by the dynamic VHC $h(q-Ls)=0$. They represent the motion of the mechanical system (1) on the constraint manifold $\bar{\Gamma}$ in (13). Their restriction to the plane $\{s=\dot{s}=0\}$ coincides with the reduced dynamics (4). The state $(\theta,\dot{\theta},s,\dot{s})\in[\mathbb{R}]_{{T_{1}}}\times\mathbb{R}\times\mathbb{R}\times\mathbb{R}$ represents global coordinates for $\bar{\Gamma}$. Using these coordinates, and with a slight abuse of notation, the closed orbit $\bar{\gamma}$ in (14) is given by

[TABLE]

This is the set we will stabilize next.

4 Step 2: Linearization along the closed orbit

The objective now is to design the control input $v$ in the extended reduced dynamics (16) so as to stabilize the closed orbit $\bar{\gamma}$ in (18). We will do so by adopting the philosophy of Hauser et al. in [17] that relies on an implicit representation of the closed orbit to derive the so-called transverse linearization along $\bar{\gamma}$. Roughly speaking, this is the linearization along $\bar{\gamma}$ of the components of the dynamics that are transversal to $\bar{\gamma}$. Hauser’s approach generalizes classical results of Hale [16, Chapter VI], requiring a moving orthonormal frame. The insight in [17] is that orthogonality is not needed, transversality is enough. This insight allowed Hauser et al. in [17] to derive a normal form analogous to that in [16, Chapter VI], but calculated directly from an implicit representation of the orbit. We shall use the same idea in the theorem below.

We begin by enhancing the results of [16, 17] in two directions. First, while [16, 17] require the knowledge of a periodic solution, we only require a parameterization of $\bar{\gamma}$ (something that is readily available in the setting of this paper, while the solution is not). Second, while [16, 17] deals with dynamics without inputs, we provide a necessary and sufficient criterion for the exponential stabilizability of the orbit.

A general result. Our first result is a necessary and sufficient condition for a closed orbit to be exponentially stabilizable. This result is of considerable practical use, and is of independent interest.

Consider a control-affine system

[TABLE]

with state $x\in\mathcal{X}$, where $\mathcal{X}$ is a closed embedded submanifold of $\mathbb{R}^{n}$, and control input $u\in\mathbb{R}^{m}$. A closed orbit $\gamma$ is exponentially stabilizable for (19) if there exists a locally Lipschitz continuous feedback $u^{\star}(x)$ such that the set $\gamma$ is exponentially stable for the closed-loop system $\dot{x}=f(x)+g(x)u^{\star}(x)$, i.e., there exist $\delta,\lambda,M>0$ such that for all $x_{0}\in\mathcal{X}$ such that $\|x_{0}\|_{\gamma}<\delta$, the solution $x(t)$ of the closed-loop system satisfies $\|x(t)\|_{\gamma}\leq M\|x_{0}\|_{\gamma}e^{-\lambda t}$ for all $t\geq 0$. Note that if $\gamma$ is exponentially stable, then $\gamma$ is asymptotically stable.

Let $T$ be a positive real number. A linear $T$-periodic system $dx/dt=A(t)x$, where $A(\cdot)$ is a continuous and $T$-periodic matrix-valued function, is asymptotically stable if all its characteristic multipliers lie in the open unit disk. A linear $T$-periodic control system

[TABLE]

where $A(\cdot)$ and $B(\cdot)$ are continuous and $T$-periodic matrix-valued functions, is stabilizable (or the pair $(A(\cdot),B(\cdot))$ is stabilizable) if there exists a continuous and $T$-periodic matrix-valued function $K(\cdot)$ such that $\dot{x}=(A(t)+B(t)K(t))x$ is asymptotically stable. In this case, we say that the feedback $u=K(t)x$ stabilizes system (20). The notion of stabilizability can be characterized in terms of the characteristic multipliers of $A(\cdot)$ (see, e.g. [3]).

Theorem 8**.**

Consider system (19), where $f$ is a $C^{1}$ vector field and $g$ is locally Lipschitz continuous on $\mathcal{X}$. Let $\gamma\subset\mathcal{X}$ be a closed orbit of the open-loop system $\dot{x}=f(x)$, and let $\vartheta\mapsto\varphi(\vartheta)$, $[\mathbb{R}]_{T}\to\mathcal{X}$, be a regular parameterization of $\gamma$. Finally, let $H:\mathcal{X}\to\mathbb{R}^{n-1}$ be an implicit representation of $\gamma$ with the properties that $H$ is $C^{1}$, $\operatorname{rank}dH_{x}=n-1$ for all $x\in H^{-1}(0)$, and $H^{-1}(0)=\gamma$.

1. (a)

The orbit $\gamma$ is exponentially stabilizable for (19) if, and only if, the linear $T$-periodic control system on $\mathbb{R}^{n-1}$

[TABLE]

is stabilizable. 2. (b)

If a $T$-periodic feedback $u=K(t)z$, with $K(\cdot)$ continuous and $T$-periodic, stabilizes the $T$-periodic system (21), then for any smooth map $\pi:\mathcal{U}\to[\mathbb{R}]_{T}$, with $\mathcal{U}$ a neighbourhood of $\gamma$ in $\mathcal{X}$ and $\pi$ such that $\pi|_{\Gamma}=\varphi^{-1}$, the feedback

[TABLE]

exponentially stabilizes the closed orbit $\gamma$ for (19).

The proof of Theorem 8 is found in Appendix B.

Remark 9**.**

Concerning the existence of the function $H$ in the theorem statement, since closed orbits of smooth dynamical systems are diffeomorphic to the unit circle $\mathbb{S}^{1}$, it is always possible to find a function $H$ satisfying the assumptions of the theorem. This well-known fact is shown, e.g., in [17, Proposition 1.2]. Since the conditions of the theorem are necessary and sufficient, the result is independent of the choice of $H$. As for the existence of the function $\pi:\mathcal{U}\to[\mathbb{R}]_{T}$, this function can be constructed by picking $\mathcal{U}$ to be a tubular neighbourhood of $\gamma$. Then there exists a smooth retraction $r:\mathcal{U}\to\gamma$. The function $\pi=\varphi^{-1}\circ r$ has the desired properties. If $\varphi(\cdot)$ is a $T$-periodic solution of $\dot{x}=f(x)$, rather than just a regular parametrization of $\gamma$, then we have $\varphi^{\prime}(t)=f(\varphi(t))$, and the scalar coefficient in (21), $\|\varphi^{\prime}(t)\|^{2}/\langle f(\varphi(t)),\varphi^{\prime}(t)\rangle$, is identically equal to one. $\triangle$

Remark 10**.**

Theorem 8 establishes the equivalence between the exponential stabilizability of the closed orbit $\gamma$ and the stabilizability of the linear periodic system (21), the so-called transverse linearization. The equivalence between these two concepts is not new, it is essentially contained in the results of [16, Chapter VI] and [17]. What is new in Theorem 8, and of considerable practical interest, is the fact that it provides an explicit expression for the transverse linearization that can be computed using any regular parametrization $\varphi$ of $\gamma$ and any implicit representation $H$ of $\gamma$ whose Jacobian matrix has full rank on $\gamma$. In contrast to the above, the methods in [16] and [17] rely on the knowledge of a periodic open-loop solution of (19) generating $\gamma$ and do not give an explicit expression for $(A(\cdot),B(\cdot))$. We also mention that Hauser’s notion of transverse linearization was applied in [32] to a special class of Euler-Lagrange systems, once again requiring the knowledge of a periodic solution. Moreover, in [30], the authors do give an explicit expressions for the transverse linearization $(A(\cdot),B(\cdot))$, but one that is only applicable to a class of Euler-Lagrange systems, while the expressions in Theorem 8 are applicable to arbitrary vector fields. $\triangle$

Remark 11**.**

In the special case of systems without control (i.e., $g(x)=0$ in (19)), Theorem 8 implies that the closed orbit $\gamma$ is exponentially stable if and only if the origin of the linear periodic system $\dot{z}=A(t)z$, with $A(t)$ given in (21), is asymptotically stable or, equivalently, multipliers of $A(t)$ have magnitude $<1$. This result is to be compared to the Poincaré stability theorem also known as the Andronov-Vitt theorem (AVT) [2] (see also [16, Chapter VI, Theorem 2.1]), stating that if $\varphi(t)$ is a $T$-periodic solution of the dynamical system $\dot{x}=f(x)$, then the closed orbit $\gamma=\operatorname{Im}(\varphi)$ is orbitally stable if the characteristic multipliers of the variational equation $\dot{x}=(df_{\varphi(t)})x$ are $\{1,\mu_{1},\ldots,\mu_{n-1}\}$, with $|\mu_{i}|<1$, $i=1,\ldots,n-1$. The link between the Andronov-Vitt theorem and Theorem 8 is that the complex numbers $\{\mu_{1},\ldots,\mu_{n-1}\}$ in the AVT are the characteristic multipliers of the $n-1\times n-1$ matrix $A(t)$ in Theorem 8. There are, however, two important differences. First, as already pointed out, Theorem 8 provides a means to directly calculate $\{\mu_{1},\ldots,\mu_{n-1}\}$ without the need to know a periodic solution of the differential equation $\dot{x}=f(x)$. Rather, any parametrization $\varphi(\vartheta)$ of $\gamma$ suffices. Furthermore, even if a periodic solution is available, the transverse linearization $A(t)$ in (21) differs from the one in the literature (see equation (1.9) in [16, Chapter VI]) because it does not rely an orthonormal moving frame666For the method of orthonormal moving frames, the reader may also consult [33]. (the matrix $Z(\vartheta)$ in [16]). Indeed, the columns of the differential of $H$ appearing in the definition of $A(t)$ span the plane orthogonal to the tangent vector to the orbit, $\varphi^{\prime}(t)$, but they do not necessarily form an orthonormal frame.

Finally, in the context of stability of orbits of dynamical systems, we mention the work of Demidovich in [8] which generalized the work of Andronov-Vitt for the stability of not necessarily closed orbits, and the work in [24] which further generalized Demidovich’s work. When specialized to closed orbits, the results in [8, 24] differ from Theorem 8 in the same way that the AVT does. $\triangle$

Design of the $T$-periodic feedback matrix $K$. Once it is established that the pair $(A(\cdot),B(\cdot))$ in (21) is stabilizable, the design of the $T$-periodic feedback matrix $K(\cdot)$ in part (b) of the theorem can be carried out by solving the periodic Riccati equation for a $T$-periodic $\Pi:\mathbb{R}\to\mathbb{R}^{n-1\times n-1}$:

[TABLE]

where $R(\cdot)=R(\cdot)^{\top}$ is a positive definite continuous $T$-periodic matrix-valued function and $Q(\cdot)=Q(\cdot)^{\top}$ is a positive definite continuous $T$-periodic matrix-valued function, and setting

[TABLE]

Theorem 6.5 in [3] states that if, and only if, $(A(\cdot),B(\cdot))$ is stabilizable and $(Q^{1/2}(\cdot),A(\cdot))$ is detectable (this latter condition is satisfied, e.g., by letting $Q$ be the identity matrix) then the Riccati equation (23) has a unique positive semidefinite $T$-periodic solution $\Pi(\cdot)$ and the feedback $u=K(t)z$, with $K(\cdot)$ given in (24), stabilizes system (21). Once this is done, the feedback $u^{\star}(x)$ in (22) exponentially stabilizes the closed orbit $\gamma$.

Application to extended reduced dynamics. We now apply Theorem 8 to the extended reduced dynamics (16) with the objective of stabilizing the closed orbit $\bar{\gamma}$ in (18). Here we have $x=(\theta,\dot{\theta},s,\dot{s})$, and the implicit representation of $\bar{\gamma}$

[TABLE]

Leveraging the parameterizations of closed orbits presented in Section 1, the parameterization of $\bar{\gamma}$ in $(\theta,\dot{\theta},s,\dot{s})$-coordinates has the form $[\mathbb{R}]_{{T_{2}}}\to[\mathbb{R}]_{{T_{1}}}\times\mathbb{R}\times\mathbb{R}\times\mathbb{R}$, $\vartheta\mapsto(\varphi_{1}(\vartheta),\varphi_{2}(\vartheta),0,0)$, with $\varphi_{1}$, $\varphi_{2}$ given by (8) and ${T_{2}}={T_{1}}$ if $\gamma$ is a rotation, and by (9) and ${T_{2}}=2\pi$ if $\gamma$ is an oscillation. Applying Theorem 8 to system (16), we get the following ${T_{2}}$-periodic linear system

[TABLE]

where

[TABLE]

Assuming that system (25) is stabilizable, then we may find the unique positive semidefinite solution of the periodic Riccati equation (23) to get the matrix-valued function $K(\cdot)$ in (24). Theorem 8 guarantees that the controller

[TABLE]

exponentially stabilizes the orbit $\bar{\gamma}$ in (18) for the extended reduced dynamics (16).

It remains to find an explicit expression for the map $\pi$. If $\gamma$ is a rotation, then in light of the parameterization (8), we may set

[TABLE]

Else, if $\gamma$ is an oscillation, using (9) we set

[TABLE]

where $\text{atan2}(\cdot,\cdot)$ is the four-quadrant arctangent function such that $\text{atan2}(\sin(\alpha),\cos(\alpha))=\alpha$ for all $\alpha\in(-\pi,\pi)$.

5 Step 3: Solution of the VHC-based orbital stabilization problem

In Section 3, we designed the feedback $\tau^{\star}$ in (15) to asymptotically stabilize the constraint manifold $\bar{\Gamma}$ associated with the dynamic VHC $h(q-Ls)=0$. In Section 4, we designed the feedback $\bar{v}$ in (27) for the double integrator $\ddot{s}=v$ rendering the closed orbit $\bar{\gamma}$ exponentially stable relative to $\bar{\Gamma}$ (i.e., when initial conditions are on $\bar{\Gamma}$). There are two things left to do in order to solve the VHC-based orbital stabilization problem. First, in order to implement the feedback $\bar{v}$ in (27), we need to relate the variables $(\theta,\dot{\theta})$ to the state $(q,\dot{q})$. Second, we need to show that the asymptotic stability of $\bar{\Gamma}$ and the asymptotic stability of $\bar{\gamma}$ relative to $\bar{\Gamma}$ imply that $\bar{\gamma}$ is asymptotically stable.

To address the first issue, we leverage the fact that, since $h^{-1}(0)$ is a closed embedded submanifold of $\mathcal{Q}$, by [23, Proposition 6.25] there exists a neighbourhood $\mathcal{W}$ of $h^{-1}(0)$ in $\mathcal{Q}$ and a smooth retraction of $\mathcal{W}$ onto $h^{-1}(0)$, i.e., a smooth map $r:\mathcal{W}\to h^{-1}(0)$ such that $r|_{h^{-1}(0)}$ is the identity on $h^{-1}(0)$. Define $\Theta:\mathcal{W}\to[\mathbb{R}]_{T}$ as $\Theta=\sigma^{-1}\circ r$. By construction, $\Theta|_{h^{-1}(0)}=\sigma^{-1}$. In other words, for all $q\in h^{-1}(0)$, $\Theta(q)$ gives that unique value of $\theta\in[\mathbb{R}]_{T}$ such that $q=\sigma(\theta)$. Using the function $\Theta$, we now define an extension of $\bar{v}$ from $\bar{\Gamma}$ to a neighborhood of $\bar{\Gamma}$ as follows

[TABLE]

We are now ready to solve the VHC-based orbital stabilization problem.

Theorem 12**.**

Consider system (1) and let $h(q)=0$ be a regular VHC of order $n-1$. Let $\sigma:[\mathbb{R}]_{T_{1}}\to\mathcal{Q}$ be a regular parametrization of $h^{-1}(0)$ and consider the following assumptions:

1. (a)

The VHC $h(q)=0$ satisfies the stabilizability condition (3). 2. (b)

The VHC $h(q)=0$ induces Lagrangian reduced dynamics as per Proposition 5. 3. (c)

For a closed orbit $\gamma$ of the reduced dynamics given in implicit form as $\gamma=\{(\theta,\dot{\theta})\in[\mathbb{R}]_{T_{1}}\times\mathbb{R}:E(\theta,\dot{\theta})=E_{0}\}$, consider one of the regular parametrizations $[\mathbb{R}]_{{T_{2}}}\mapsto[\mathbb{R}]_{T_{1}}\times\mathbb{R}\times\mathbb{R}\times\mathbb{R}$ discussed in Section 4. Assume that the ${T_{2}}$-periodic system (25)-(26) is stabilizable.

Under the assumptions above, let $Q(\cdot)=Q(\cdot)^{\top}$ be a positive definite ${T_{2}}$-periodic $\mathbb{R}^{3\times 3}$-valued function such that $(Q^{1/2},A)$ is detectable, and pick any $R>0$. The smooth dynamic feedback

[TABLE]

with $\tau^{\star}$ defined in (15), $v^{\star}$ defined in (27), (28), and where $K(\cdot)$ in (24) results from the solution of the ${T_{2}}$-periodic Riccati equation (23), asymptotically stabilizes both sets $\bar{\gamma}\subset\bar{\Gamma}$ given in (13), (14).

{pf}

Propositions 6 and 7 establish that the feedback (15) stabilizes the set $\bar{\Gamma}$. Theorem 6.5 in [3] establishes that $K(\cdot)$ in (24) is well-defined, and Theorem 8 establishes that $v^{\star}$ in (28) stabilizes $\bar{\gamma}$ relative to $\bar{\Gamma}$. Since $\bar{\gamma}$ is a compact set, the reduction theorem for stability of compact sets in [29], [9] implies that $\bar{\gamma}$ is asymptotically stable for the closed-loop system. ∎

The block diagram of the VHC-based orbital stabilizer is depicted in Figure 4.

Remark 13**.**

In Theorem 12 we only claim asymptotic stability of $\bar{\gamma}$, even though Theorem 8 guarantees that $\bar{\gamma}$ is exponentially stable relative to $\bar{\Gamma}$. The reason is that in the proof we use a reduction theorem for asymptotic stability of sets [9]. A different proof technique could be used to show that $\bar{\gamma}$ is in fact exponentially stable for the closed-loop system. $\triangle$

6 Discussion

In this section we briefly compare the control methodology of this paper with the ones in [28, 5, 4, 32].

Comparison with [28]. The notion of dynamic hybrid extension introduced by Morris and Grizzle in [28] bears a conceptual resemblance to dynamic VHCs and their extended reduced dynamics presented in this article. In [28], the VHCs that induce stable walking gaits of biped robots are parameterized using variables whose evolution are event-triggered. In particular, the VHC parameters get updated after each impact of the swing leg with the ground. The update law is designed such that the invariance of a suitably modified manifold, which the authors call the extended zero dynamics manifold, is preserved while simultaneously enforcing a periodic stable walking gait on the biped. Our approach follows the same philosophy of preserving the invariance of a suitably modified manifold in order to maintain the desired configurations of the mechanical system. However, in our framework, the dynamics of the VHC parameter are continuous rather than event-triggered.

Comparison with [5]. The approach by Canudas-de-wit et al. in [5] also relies on dynamically changing the geometry of VHCs. A target orbit on the constraint manifold, which is generated by a harmonic oscillator, is considered and the dynamics of the VHC parameter is designed such that the target orbit is stabilized on the constraint manifold. This approach, however, cannot be used to stabilize an assigned closed orbit induced by the original VHC on the constraint manifold. Moreover, the methodology in [5] has only been employed to control the periodic motions of a pendubot. It is unclear to what extent it can be generalized to other mechanical systems.

Comparison with [4, 32, 30]. In [4, 32], the authors employ VHCs to find feasible closed orbits of underactuated mechanical systems. Once the orbit is found, it is stabilized through transverse linearization of the $2n$-dimensional dynamics (1) along the closed orbit. Similarly to this paper, in [4, 32] the stabilization of the transverse linearization is carried out by solving a periodic Riccati equation. But while the linearized system in [4, 32] has dimension $2n-1$, the linearized system (25) always has dimension $3$. And while the feedback in Theorem 12 is time-independent, the one proposed in [4, 32] is time-varying. Additionally, while the approach proposed in this paper gives explicit parametrizations of the orbits to be stabilized, the approaches in [4, 32] require the knowledge of the actual periodic trajectory which is not available in analytic form. The most important difference between the approach in this paper and the ones in [4, 32] lies in the fact that, in [4, 32], the time-varying controller does not preserve the invariance of the constraint manifold. The work in [30] generalizes the theory of [32] to systems with degree of underactuation greater than one. The philosophy in [30] is analogous to that of [32] and shares the same differences just outlined with our work. The authors use virtual constraints to help identify a desired closed orbit of the control system, then linearize the control system around said orbit to design a stabilizer. In this paper, we only deal with systems with degree of underactuation one.

We end this section with a remark about the computational cost of the controller proposed in Theorem 12. The controller has two components: an input-output feedback linearizing controller, $\tau^{\star}$, enforcing the dynamic VHC, and a scalar feedback, $v^{\star}$, for the double-integrator stabilizing the desired closed orbit. The computational cost of these controllers for real-time implementation is essentially equivalent to that of virtual constraint controllers used by Grizzle and collaborators for biped robots and by many other researchers in the area. The design of the orbit stabilizer $v^{\star}$ involves the solution of a periodic Riccati equation for the three-dimensional linear periodic system (25). The dimension of this problem is always $3$, independent of the number of DOFs of the original mechanical system. As described above, this is a major advantage of the simultaneous stabilization method proposed in this paper. We surmise that the proposed approach can be particularly effective to reduce the design complexity for robots with a large number of DOFs.

7 Example

In this section we use the theory developed in this paper to enhance a result found in [7]. We consider the model of a V/STOL aircraft in planar vertical take-off and landing mode (PVTOL), introduced by Hauser et al. in [18]. The vehicle in question is depicted in Figure 5, where it is assumed that a preliminary feedback has been designed making the centre of mass of the aircraft lie on a unit circle on the vertical plane, \mathcal{C}=\{x\in\mathbb{R}^{2}:\,\big{|}x\big{|}=1\}, also depicted in the figure. In [7] it was shown that the model of the aircraft on the circle is given by

[TABLE]

where $q_{1}$ denotes the roll angle, $q_{2}$ the angular position of the aircraft on the circle, and $u$ the so-called tangential control input resulting from the design in [7]. Also, $\mu$ and $\epsilon$ are positive constants. In this example, we set $\mu/\epsilon=1$.

In [7], a feedback $u(q,\dot{q})$ was designed to enforce a regular VHC of the form $h(q)=q_{1}-f(q_{2})=0$, represented in Figure 5. It was shown that the ensuing reduced dynamics, a few orbits of which are depicted in Figure 6, are Lagrangian. Each closed orbit in Figure 6 represents a motion of the PVTOL on the circle, with roll angle $q_{1}$ constrained to be a function of the position, $q_{2}$, on the circle. Orbits in the shaded area represent a rocking motion of the PVTOL along the circle (these are oscillations), while orbits in the unshaded area represent full traversal of the circle (these are rotations). The theory in [7] was unable to stabilize individual closed orbits of the reduced dynamics. The theory of this paper fills the gap left open in [7].

We wish to stabilize the closed orbit $\gamma^{+}$ depicted in Figure 6 which corresponds to the energy level set $E_{0}=41.5$. The parametrization of $\gamma^{+}$ on the $(q_{2},\dot{q}_{2})$ plane is $\vartheta\mapsto(\vartheta,\sqrt{2/M(\vartheta)(E_{0}-V(\vartheta))})$. Here, $T_{1}=T_{2}=2\pi$. We render the VHC dynamic by setting $h^{s}(q)=q_{1}-L_{1}s-f(q_{2}-L_{2}s)=0$, with $L=\operatorname{col}(L_{1},L_{2})=\operatorname{col}(1,1)$. We enforce this dynamic VHC by means of the feedback $\tau^{\star}$ in (15), with $k_{p}=100$ and $k_{d}=10$. Thus trajectories converge to $\bar{\Gamma}$ at a rate of $\exp(-5t)$. This rate of convergence is chosen so as to make the enforcement of the dynamic VHC faster than the orbit stabilization mechanism.

Since $\gamma^{+}$ is a rotation, we parameterize it with the map (8). We check numerically that the $2\pi$-periodic pair $(A(t),B(t))$ in (25), (26) is controllable, and after some tuning we pick $R=400$ and $Q=\operatorname{diag}\{1/2,\,10^{4},\,1\}$ to set up the Riccati equation (23). We numerically solve this equation using the one-shot generator method [19] (see also [21, 15] for a detailed treatment of existing numerical algorithms to solve the periodic Riccati equation) and find the gain matrix $K(\cdot)$. The resulting characteristic multipliers of the transverse linearization (21) with time-varying feedback $u=K(t)z$ are $\{0.0447,-3.6816\times 10^{-5}\pm 2.7122\times 10^{-5}i\}$. This means that trajectories on the constraint manifold near $\bar{\gamma}$ converge to $\bar{\gamma}$ at a rate of $\exp[\log(0.0447)t/(2\pi)]=\exp(-0.49t)$. Thus the enforcement of the dynamic VHC occurs faster than the orbit stabilization mechanism.

The simulation results for the controller in Theorem 12 are presented next. We pick the initial condition $q(0)=(0,\pi/2+0.2)$, $\dot{q}(0)=(0,0)$, $(s(0),\dot{s}(0))=(0,0)$. We verified that other initial conditions in a neighborhood of $\bar{\Gamma}$ give similar results as the ones that follow. Figures 7 and 8 depict the graph of the function $h^{s(t)}(q(t))$ and the output of the double integrator, $s(t)$, respectively. They reveal that the VHC is properly enforced and that $s(t)\to 0$. Figures 9 and 10 depict the energy of the vehicle on the constraint manifold and the time trajectory of $(\theta(t),\dot{\theta}(t))$ on the cylinder $\mathbb{S}^{1}\times\mathbb{R}$. The energy level $E_{0}$ is stabilized and the trajectory on the cylinder converges to $\gamma^{+}$. Finally, Figure 11 depicts the graph of the roll angle $q_{1}(t)$, demonstrating that, due to the enforcement of the dynamic version of the VHC depicted in Figure 5, the vehicle does not roll over for the given initial condition. When comparing Figure 7 with Figures 8 and 9, it is evident that the enforcement of the dynamic VHC occurs faster than the convergence to the closed orbit. As a final remark, in the proposed framework the roots of the polynomial $s^{2}+k_{d}s+k_{p}$ determine the rate of convergence of trajectories to the constraint manifold $\bar{\Gamma}$, while the characteristic multipliers concern the constrained dynamics on $\bar{\Gamma}$, and they characterize the rate at which trajectories on $\bar{\Gamma}$ converge to the closed orbit $\bar{\gamma}$.

8 Conclusions

We have proposed a technique to enforce a VHC on a mechanical control system and simultaneously stabilize a closed orbit on the constraint manifold. The theory of this paper is applicable to mechanical control systems with degree of underactuation one. For higher degrees of underactuation, the reduced dynamics are described by a differential equation of order higher than two and, generally, the problem of characterizing closed orbits becomes harder. The result of Section 4 concerning the exponential stabilization of closed orbits for control-affine systems is still applicable in this case.

Appendix A Proofs of Technical Results in Section 3

Proof of Proposition 6. Considering the output $e=h(q-Ls)$ and taking two derivatives along system (10), we get

[TABLE]

where $A^{s}(q)=dh_{q-Ls}D^{-1}(q)B(q)$. Denote $\mu^{s}:=\min_{q\in(h^{s})^{-1}(0)}\det A^{s}(q)$. Then, $s\mapsto\mu^{s}$ is a continuous function. We claim that $\mu^{0}\neq 0$. Indeed, the assumption that $h(q)=0$ is regular implies by Proposition 1 that $\det A^{0}(q)\neq 0$ for all $q\in(h^{0})^{-1}(0)=h^{-1}(0)$. Since $h^{-1}(0)$ is a compact set and $q\mapsto\det A^{0}(q)$ is continuous, $\min(\det A^{0}(q))\neq 0$, proving that $\mu^{0}\neq 0$, as claimed. By continuity, there exists an open interval $\mathcal{I}\subset\mathbb{R}$ containing $s=0$ such that $\mu^{s}\neq 0$ on $\mathcal{I}$ implying that $A^{s}(q)$ is nonsingular for all $q\in(h^{s})^{-1}(0)$ and all $s\in\mathcal{I}$. ∎

Proof of Proposition 7. By (13), we have $\bar{\Gamma}=\{(q,\dot{q},s,\dot{s})\in T\bar{\mathcal{Q}}:(q-Ls,\dot{q}-L\dot{s})\in\Gamma\}$, from which it follows that $\|(q,\dot{q},s,\dot{s})\|_{\bar{\Gamma}}=\|(q-Ls,\dot{q}-L\dot{s})\|_{\Gamma}$. This fact and the inequalities in (3) imply that

[TABLE]

Letting $e=h(q-Ls)$, the feedback (15) gives $\ddot{e}+k_{d}\dot{e}+k_{p}e=0$, so that the equilibrium $(e,\dot{e})=(0,0)$ is asymptotically stable. Since $(e,\dot{e})=H(q-Ls,\dot{q}-L\dot{s})$, property (30) implies that $\bar{\Gamma}$ is asymptotically stable. ∎

Appendix B Proof of Theorem 8

Let $H:\mathcal{X}\to\mathbb{R}^{n-1}$ and $\pi:\mathcal{U}\to[\mathbb{R}]_{T}$ be as in the theorem statement. We claim that there exists a neighborhood $\mathcal{V}$ of $\gamma$ in $\mathcal{X}$ such that the map $F:\mathcal{V}\to[\mathbb{R}]_{T}\times\mathbb{R}^{n-1}$, $x\mapsto(\vartheta,z)=(\pi(x),H(x))$ is a diffeomorphism onto its image. By the generalized inverse function theorem [14], we need to show that $dF_{x}$ is an isomorphism for each $x\in\gamma$, and that $F|_{\gamma}$ is a diffeomorphism $\gamma\to[\mathbb{R}]_{T}\times\{0\}$. The first property was proved in [17, Proposition 1.2]. For the second property, we observe that $F|_{\gamma}=\pi|_{\gamma}\times\{0\}$ is a diffeomorphism $\gamma\to[\mathbb{R}]_{T}\times\{0\}$, since $\pi|_{\gamma}=\varphi^{-1}$ is a diffeomorphism $\gamma\to[\mathbb{R}]_{T}$. The smooth inverse of $F|_{\gamma}$ is

[TABLE]

Thus $F:\mathcal{V}\to[\mathbb{R}]_{T}\times\mathbb{R}^{n-1}$ is a diffeomorphism onto its image, as claimed. Since $\vartheta\mapsto\varphi(\vartheta)$ is a regular parameterization of the orbit $\gamma$, and since $\gamma$ is an invariant set for the open-loop system, $f(\varphi(\vartheta))$ is proportional to $\varphi^{\prime}(\vartheta)$. More precisely, defining the continuous function $[\mathbb{R}]_{T}\to\mathbb{R}$,

[TABLE]

we have that

[TABLE]

and $\rho$ is bounded away from zero. We now represent the control system (19) in $(\vartheta,z)$ coordinates. The development is a slight variation of the one presented in the proof of [17, Proposition 1.4], the variation being due to the fact that, in [17], it is assumed that $\rho=1$. For the $\vartheta$-dynamics, we have

[TABLE]

We claim that the restriction of the drift term to $\gamma$ is $\rho(\vartheta)$. Indeed, using (31) and (33), we have

[TABLE]

The last equality is due to the fact that $\pi(\varphi(\vartheta))=\vartheta$, so that $d\pi_{\varphi(\vartheta)}\varphi^{\prime}(\vartheta)=1$. Thus we may write

[TABLE]

where $f_{1}(\vartheta,0)=0$. The derivation of the $z$ dynamics is essentially the same as in [17, Proposition 1.4] so we present their form without proof. The control system (19) in $(\vartheta,z)$ coordinates has the form

[TABLE]

where $f_{1}$ and $f_{2}$ satisfy $f_{1}(\vartheta,0)=0$, $f_{2}(\vartheta,0)=0$, $\partial_{z}f_{2}(\vartheta,0)=0$.

Letting {\tilde{T}}=\int_{0}^{T}\big{|}1/\rho(u)\big{|}du, we have that ${\tilde{T}}>0$ because $\rho$ is bounded away from zero. Consider the partial coordinate transformation $\tau:[\mathbb{R}]_{T}\to[\mathbb{R}]_{{\tilde{T}}}$ defined as

[TABLE]

Since $\rho$ is bounded away from zero, the derivative $\tau^{\prime}(\vartheta)$ is also bounded away from zero, implying that $\tau$ is a diffeomorphism. We denote by $\vartheta(\tau)$ the inverse of $\tau(\vartheta)$. System (34) in $(\tau,z)$ coordinates reads as

[TABLE]

where $\tilde{f}_{1}(\tau,z)=f_{1}(\vartheta(\tau),z)/\rho(\vartheta(\tau))$, $\tilde{g}_{1}(\tau,z)=g_{1}(\vartheta(\tau),z)/\rho(\vartheta(\tau))$, and $\tilde{f}_{2}(\tau,z)=f_{2}(\vartheta(\tau),z)$.

System (35) has the same form of that in [17, Proposition 1.4] (which, however, has no control inputs). By [17, Proposition 1.5], we deduce that the orbit $\gamma$ is exponentially stabilizable if and only if the ${\tilde{T}}$-periodic system

[TABLE]

is stabilizable. Since $\vartheta(\tau)$ is a diffeomorphism, we may perform the time-scaling

[TABLE]

Thus $\gamma$ is exponentially stabilizable if and only if the $T$-periodic system (36) is asymptotically stable. By comparing the system and input matrices of (36) with those of system (21), we see that to prove part (a) of Theorem 8 it suffices to show that

[TABLE]

Since $z=H(x)$, the coefficient of $u$ in $\dot{z}$ is

[TABLE]

Using (31) we get $g_{2}(\vartheta,0)=L_{g}H\circ F^{-1}(\vartheta,0)=L_{g}H(\varphi(\vartheta))$. This proves identity (38).

Concerning identity (37), and referring to system (34), $\bar{A}(\vartheta)$ is the Jacobian of $\dot{z}$ with respect to $z$ evaluated at $(z,u)=(0,0)$. Since

[TABLE]

we have

[TABLE]

By the chain rule and the identity (31), we get

[TABLE]

To show that identity (37) holds, we need to show that \partial_{z}F^{-1}(\vartheta,z)\big{|}_{z=0}=dH_{\varphi(\vartheta)}^{\dagger}. To this end, we use the fact that

[TABLE]

or

[TABLE]

In light of the above, \partial_{z}F^{-1}(\vartheta,z)\big{|}_{z=0} is uniquely defined by the identities

[TABLE]

so we need to show that

[TABLE]

Identity (40) holds by virtue of the fact that $dH^{\dagger}$ is the right-inverse of $dH$. Using the definition of pseudoinverse and taking the transpose of (39), we may rewrite (39) as

[TABLE]

Since $\varphi(\pi(x))=x$ for all $x\in\gamma$, we have $d\varphi_{\pi(x)}d\pi_{x}=I_{n}$, or

[TABLE]

so that

[TABLE]

Since $H(\varphi(\vartheta))\equiv 0$, $dH_{\varphi(\vartheta)}d\varphi_{\vartheta}=0$ for all $\vartheta\in[\mathbb{R}]_{T}$. Thus, $dH_{\varphi(\vartheta)}d\pi_{\varphi(\vartheta)}^{\top}=0$ for all $\vartheta\in[\mathbb{R}]_{T}$.

We have thus shown that identities (39) and (40) hold, implying that identity (37) holds. This concludes the proof of part (a) of the theorem.

For part (b), let $A(t)$, $B(t)$ be as in (21), and suppose that the origin of $\dot{z}=(A(t)+B(t)K(t))z$ is asymptotically stable. With the controller $u^{\star}(x)=K(\pi(x))H(x)$, the dynamics of the closed-loop system in $(\tau,z)$ coordinates read as

[TABLE]

For the $z$ dynamics we have

[TABLE]

with $\tilde{F}_{2}(\vartheta,0)=0$, $\partial_{z}\tilde{F}_{2}(\vartheta,0)=0$. By using $\vartheta$ as time variable, the linear part of the $z$-dynamics reads as

[TABLE]

Using the identities (37) and (38) we rewrite the above as

[TABLE]

By assumption, the origin of this system is asymptotically stable, implying that the origin of the system $\dot{z}=[\bar{A}(\vartheta(\tau))+g_{2}(\vartheta(\tau),0)K(\vartheta(\tau))]z$ has the same property. Referring to (42) and using [17, Proposition 1.5], we conclude that the closed orbit $\gamma$ is exponentially stable for the closed-loop system (41) and hence also for system (19) with feedback $u^{\star}(x)=K(\pi(x))H(x)$. ∎

{ack}

A. Mohammadi and M. Maggiore were supported by the Natural Sciences and Engineering Research Council (NSERC) of Canada. A. Mohammadi was partially supported by the University of Toronto Doctoral Completion Award (DCA).