Advances in Stabilisation of Hybrid Stochastic Differential Equations by Delay Feedback Control

TL;DR

This paper introduces a new method for designing delay feedback control in hybrid stochastic differential equations, linking it to existing control functions without delay, and provides bounds on delay length with numerical validation.

Contribution

It presents a novel theorem connecting delay feedback control with non-delay control functions for HSDEs, facilitating stability analysis and control design.

Findings

Established a new theorem linking delay feedback control to existing control functions.

Derived methods to determine upper bounds for delay length.

Numerical simulations validate the theoretical results.

Abstract

A novel approach to design the feedback control based on past states is proposed for hybrid stochastic differential equations (HSDEs). This new theorem builds up the connection between the delay feedback control and the control function without delay terms, which enables one to construct the delay feedback control using the existing results on stabilities of HSDEs. Methods to find the upper bound of the length of the time delay are also investigated. Numerical simulations are presented to demonstrate the new theorem.

Equations

$$dx(t)=f(x(t),r(t),t)dt+g(x(t),r(t),t)dB(t).$$

$$dx(t)=[f(x(t),r(t),t)+u(x(t),r(t),t)]dt+g(x(t),r(t),t)dB(t)$$

$$dx(t)=[f(x(t),r(t),t)+u(x(t-\epsilon ),r(t),t)]dt+g(x(t),r(t),t)dB(t).$$

$$dx(t)=[f(x(t),r(t),t)+u(x(t-\tau ),r(t),t)]dt+g(x(t),r(t),t)dB(t)$$

$$dx(t)$$

$$-[u(x(t),r(t),t)-u(x(t-\tau ),r(t),t)]dt.$$

$$\mathbb{P}\{r(t+\Delta )=j|r(t)=i\}=\{\begin{array}{ll}{\gamma }_{ij}\Delta +o(\Delta )& ifi\ne j,\\ 1+{\gamma }_{ii}\Delta +o(\Delta )& ifi=j,\end{array}$$

$${\gamma }_{ii}=-\sum _{j\ne i}{\gamma }_{ij}.$$

$$\{x(\theta ):-\tau \le u\le 0\}=\phi \in C([-\tau ,0];R^n)andr(0)=r_0\in S.$$

$$|f(x,i,t)-f(y,i,t)|\le L_1|x-y|,$$

$$|u(x,i,t)-u(y,i,t)|\le L_2|x-y|,$$

$$|g(x,i,t)-g(y,i,t)|\le L_3|x-y|$$

$$f(0,i,t)=0,g(0,i,t)=0,u(0,i,t)=0$$

$$|f(x,i,t)|\le L_1|x|,|u(x,i,t)|\le L_2|x|,|g(x,i,t)|\le L_3|x|$$

$$\mathbb{E}\Vert x_t{\Vert }^q<\infty for\; allt\ge 0\; and\; anyq>0,$$

$$x(t;\phi ,r_0,0)=x(t;x_{t_0},r(t_0),t_0)andr(t;r_0,0)=r(t;r(t_0),t_0)$$

$$dy(t)=\big{(}f(y(t),r(t),t)+u(y(t),r(t),t)\big{)}dt+g(y(t),r(t),t)dB(t).$$

$$\mathbb{E}|y(t;y(t_0),r(t_0),t_0){|}^p\le M\mathbb{E}|y(t_0){|}^p{e}^{-\gamma (t-t_0)}\forall t\ge t_0\ge 0$$

$$\underset{t_0\le t\le t_0+T+\tau }{\sup }\mathbb{E}|x(t){|}^p\le K_1\mathbb{E}\Vert x_{t_0}{\Vert }^p,$$

$$\mathbb{E}\Big{(}\sup_{t_{0}\leq t\leq t_{0}+T+\tau}|x(t)|^{p}\Big{)}\leq K_{2}\mathbb{E}\|x_{t_{0}}\|^{p},$$

$$\sup_{t_{0}\leq t\leq t_{0}+T}\mathbb{E}\Big{(}\sup_{0\leq u\leq\tau}|x(t+u)-x(t)|^{p}\Big{)}\leq K_{3}\mathbb{E}\|x_{t_{0}}\|^{p},$$

$$K_1$$

$$K_2$$

$$\displaystyle=\left\{\begin{array}[]{ll}4^{p-1}K_{1}(p,\tau,T)\big{(}1+(L_{1}^{p}+L_{2}^{p})(T+\tau)^{p}+C_{p}L_{3}^{p}(T+\tau)^{0.5p}\big{)},&\hbox{if }p\geq 2,\\ \big{(}4K_{1}(2,\tau,T)\big{[}(L_{1}^{2}+L_{2}^{2})(T+\tau)^{2}+L_{3}^{2}(T+\tau)\big{]}\big{)}^{p/2},&\hbox{if }p\in(0,2),\end{array}\right.$$

$$K_3$$

$$\mathbb{E}|x(t){|}^p$$

$$\le$$

$$\int_{t_{0}}^{t}\mathbb{E}|x(s-\tau)|^{p}ds\leq\tau\mathbb{E}\|x_{t_{0}}\|^{p}+\int_{t_{0}}^{t}\Big{(}\sup_{t_{0}\leq u\leq s}\mathbb{E}|x(u)|^{p}\Big{)}ds.$$

$$\displaystyle\mathbb{E}|x(t)|^{p}\leq(1+\tau)\mathbb{E}\|x_{t_{0}}\|^{p}+p\big{[}L_{1}+L_{2}+0.5(p-1)L_{3}\big{]}\int_{t_{0}}^{t}\Big{(}\sup_{t_{0}\leq u\leq s}\mathbb{E}|x(u)|^{p}\Big{)}ds.$$

$$\sup_{t_{0}\leq u\leq t}\mathbb{E}|x(u)|^{p}\leq(1+\tau)\mathbb{E}\|x_{t_{0}}\|^{p}+p\big{[}L_{1}+L_{2}+0.5(p-1)L_{3}\big{]}\int_{t_{0}}^{t}\Big{(}\sup_{t_{0}\leq u\leq s}\mathbb{E}|x(u)|^{p}\Big{)}ds.$$

$$\underset{t_0\le u\le t_0+T}{\sup }\mathbb{E}|x(u){|}^p\le (1+\tau )e^{p(T+\tau )[L_1+L_2+0.5(p-1)L_3]}\mathbb{E}\Vert x_{t_0}{\Vert }^p.$$

$$\mathbb{E}(|x(t){|}^2|{\mathcal{F}}_{t_0})\le (1+\tau )e^{2(T+\tau )(L_1+L_2+0.5L_3^2)}\Vert x_{t_0}{\Vert }^2$$

$$\displaystyle\mathbb{E}(|x(t)|^{p}|{\cal F}_{t_{0}})\leq\big{(}\mathbb{E}(|x(t)|^{2}|{\cal F}_{t_{0}})\big{)}^{p/2}\leq(1+\tau)^{p/2}e^{p(T+\tau)(L_{1}+L_{2}+0.5L_{3}^{2})}\|x_{t_{0}}\|^{p}.$$

Full text

Key words. Brownian motion, Markov chain, exponential stability, delay feedback control.

AMS subject classifications. 60H10, 60J10, 93D15.

1 Introduction

Many systems in the real word may experience abrupt changes in their structures and parameters due to sudden changes of system factors, for example, a failure of a power station in a network, a change of interest rate in an economic system, an environmental change in an ecological system. Hybrid systems driven by continuous-time Markov chains have been used widely to model these systems (see, e.g., [2, 4, 7, 18, 20, 21]).

One important class of hybrid systems is the hybrid stochastic differential equations (SDEs; also known as SDEs with Markovian switching). Hybrid SDEs are in general described by

[TABLE]

Here the state $x(t)$ takes values in $R^{n}$ and the mode $r(t)$ is described by a Markov chain taking values in a finite space $S=\{1,2,\cdots,N\}$, $B(t)$ is a Brownian motion, $f$ and $g$ are referred to as the drift and diffusion coefficient, respectively. (Further details on the notation will be stated in Section 2.) One of the important issues in the study of hybrid SDEs is the analysis of stability. For example, Ji et al. [6], and Mariton [21] studied the stability of the jump linear systems. Basak et al. [3] discussed the stability of semi-linear hybrid stochastic differential equations (SDEs) while Mao [12] investigated the stability of a nonlinear hybrid SDEs. Shaikhet [24] took the time delay into account and considered the stability of semi-linear hybrid SDEs with delay, while Mao et al. [18] investigated the stability of a nonlinear hybrid SDEs with delay. Taking into account of the parameter uncertainty, Mao [14] studied the stability of hybrid stochastic delay interval systems. There is now an intensive literature in the area of hybrid SDEs (for further references see, e.g., [13, 15, 17, 19, 25, 26, 27, 29]).

Given an $n$-dimensional unstable hybrid SDE in the form of (1.1), it is classical to find a feedback control $u(x(t),r(t),t)$, based on the current state $x(t)$, for the controlled system

[TABLE]

to become stable. This stabilisation problem by (non-delay) state feedback controls has been well studied (see, e.g., [19, 20, 28]).

On the other hand, there is always a time lag between the time when the observation of the state is made and the time when the feedback control reaches the system. (It takes 1.28 seconds for a radio signal from the moon to reach the earth.) Traditionally, we usually assume that the time lag is extremely small (namely the feedback control acts extremely fast). Nevertheless, there is a delay, say $\varepsilon$. So the real controlled system should be in the form of

[TABLE]

For example, the time unit is of year while the time lag $\varepsilon$ is a microsecond ($=3.17*10^{-14}$ year) or a nanosecond ($=3.17*10^{-17}$ year). It seems that it has been taken as a grant that if the controlled system (1.2) is stable, so is (1.3) provided $\varepsilon$ is sufficiently small. However, there is a counter example which shows that the controlled system (1.2) is exponentially stable in $p$th moment, but the corresponding system (1.3) is NOT no matter how small $\varepsilon$ is (please see the Appendix).

More usefully, the time lag may NOT be extremely small. In practice, to reduce the control cost, it may be more implementable to allow the feedback control to act reasonably fast but not necessarily extremely fast. Let $\tau$ denote the time lag between the time when the observation of the state is made and the time when the feedback control reaches the system. It is then certainly more realistic that the control should depend on the past state $x(t-\tau)$. Accordingly, the control should be of the form $u(x(t-\tau),r(t),t)$. Consequently, the stabilisation problem becomes to design a delay feedback control $u(x(t-\tau),r(t),t)$ for the controlled system

[TABLE]

to become stable. Mao, Lam and Huang [16] were the first to study this stabilisation problem by the delay feedback control for hybrid SDEs, although the method of delay feedback controls has been well used in the area of ordinary differential equations (see, e.g., [1, 5, 23]). The main idea in [16] was to use the theory of linear matrix inequalities (LMIs) along with the method of Lyapunov functionals to design the linear delay feedback control in the form of $u(x(t-\tau),r(t),t)=F(r(t))G(r(t))x(t-\tau)$, where $F:S\to R^{n\times l}$ and $G:S\to R^{l\times n}$. They discussed two cases: (i) state feedback, namely design $F(\cdot)$ when $G(\cdot)$ was given; (ii) output injection, namely design $G(\cdot)$ when $F(\cdot)$ was given. The advantage of the results in [16] lies in that either $F(\cdot)$ or $G(\cdot)$ can be solved efficiently by the technique of LMIs. The disadvantage is that the given unstable hybrid SDE should be either linear or nonlinear but dominated by a linear system. Such a limitation is unavoidable due to the key technique of LMIs used.

In this paper we will take a completely new approach in order to avoid this limitation. As we mentioned before, the stabilization by the (non-delay) feedback control (namely, the controlled system (1.2) has been well studied. In other words, there are lots of results on how to design the control function $u(x,i,t)$ to make the controlled system ( 1.2) stable.

• Question: *Can we make use of this same control function to make the delay feedback controlled system (1.4) stable? *

Should the answer is yes, this would be great as the stabilization problem (1.4) by a delay feedback control could be transferred to the well-known classical stabilization problem (1.2) by a non-delay feedback control. Is this possible? To see the possibility, we rearrange the controlled system (1.4) as

[TABLE]

Comparing this with (1.2), we may regard it (i.e., system (1.4)) as the perturbed system of (1.2) with the perturbation $-[u(x(t),r(t),t)-u(x(t-\tau),r(t),t)]dt$. If the time lag $\tau$ (the duration between the time when the state observation is made and the time when the feedback control reaches the system) is sufficiently small (namely, the feedback control acts sufficiently fast) while the control function $u(x,i,t)$ is globally Lipschitz continuous in $x$, then the perturbation might be sufficiently small so that system (1) should perform in a similar way as system (1.2) does (namely, stable). It is this perturbation idea that motivates us to show in this paper the following result:

• Answer: *Under the global Lipschitz condition on the system coefficients $f,g$ and the control function $u$, if the control function $u$ makes the controlled system (1.2) to be exponentially stable in the $p$th moment ($p>0$) then there is a positive number $\tau^{*}$, which can be determined numerically, such that the same control function $u$ will also make the controlled system (1.4) to be exponentially stable in the $p$th moment as long as the delay feedback control $u(x(t-\tau),r(t),t)$ acts sufficiently fast in the sense $\tau<\tau^{*}$. *

Let us highlight a couple of important features of this new result:

• •

The result covers a much wider class of nonlinear hybrid SDEs than Mao et al. [16].

• •

The delay feedback stabilisation problem (1.4) is transferred to the classical stabilisation problem (1.2) so that many existing results and techniques can be used to design the required control function $u(x,i,t)$.

• •

The positive number $\tau^{*}$ can be determined numerically, which means our theory can be implemented easily.

• •

More importantly, our new result gives a theoretical support for the general practice of the non-delay state feedback control. In practice, there is always a time lag, though it might be extremely small, from the moment when the state is observed to the time when the state feedback control reaches the system. In other words, the controlled system in practice should be in the form of (1.3). In the past, we have always designed the control function to make the controlled system (1.2) to be stable while required the feedback control to act extremely quickly (like a non-delay though there is a delay), namely $\varepsilon$ is extremely small whence less than $\tau^{*}$ (it seems that the physical existence of $\tau^{*}$ was not known before). Our new theory explains why this general practice has worked for so many years (under the global Lipschitz condition of course).

• •

Mao et al. [16] only studied the stabilisation in the mean square exponential stability. We here discuss the stabilisation in more general $p$th exponential stability (for $p>0$). This is particularly significant when $p\in(0,1)$ where the stabilisation effect of Brownian motions could be used (see Example 5.3 for more details please).

Let us begin to establish our new theory. We will state some preliminaries in Section 2 and present a number of lemmas in Section 3. We will prove the main results of this paper in Section 4 while demonstrate how our new results can be implemented easily in Section 5 making use of the existing theory on the non-delay state feedback controls. We will conclude our paper finally in Section 6.

2 Preliminaries

Throughout this paper, unless otherwise specified, we will use the following notation. If $A$ is a vector or matrix, its transpose is denoted by $A^{T}$. If $x\in R^{n}$, then $|x|$ is its Euclidean norm. If $A$ is a matrix, we let $|A|=\sqrt{\hbox{\rm trace}(A^{T}A)}$ be its trace norm. iIf both $a,b$ are real numbers, then $a\vee b=\min\{a,b\}$ and $a\wedge b=\max\{a,b\}$.

Let $(\Omega,{\cal F},\{{\cal F}_{t}\}_{t\geq 0},\mathbb{P})$ be a complete probability space with a filtration $\{{\cal F}_{t}\}_{t\geq 0}$ satisfying the usual conditions (i.e. it is right continuous and ${\cal F}_{0}$ contains all $\mathbb{P}$-null sets). If $A$ is a subset of $\Omega$, denote by $I_{A}$ its indicator function; that is $I_{A}(\omega)=1$ when $\omega\in A$ and [math] otherwise. Let $B(t)=(B_{1}(t),\cdots,B_{m}(t))^{T}$ be an $m$-dimensional Brownian motion defined on the probability space. Let $r(t)$, $t\geq 0$, be a right-continuous Markov chain on the same probability space taking values in a finite state space $S=\{1,2,\cdots,N\}$ with its generator $\Gamma=(\gamma_{ij})_{N\times N}$ given by

[TABLE]

where $\Delta>0$. Here $\gamma_{ij}\geq 0$ is the transition rate from $i$ to $j$ if $i\not=j$ while

[TABLE]

We assume that the Markov chain $r(\cdot)$ is independent of the Brownian motion $B(\cdot)$. Moreover, denote by $M_{{\cal F}_{t}}(S)$ the family of all ${\cal F}_{t}$-measurable $S$-valued random variables.

Let $\tau$ be a positive number. Denote by $C([-\tau,0];R^{n})$ the family of continuous functions $\varphi:[-\tau,0]\to R^{n}$ with the norm $\|\varphi\|=\sup_{-\tau\leq u\leq 0}|\varphi(u)|$. For $q>0$ and $t\geq 0$, denote by $L^{p}_{{\cal F}_{t}}(C)$ the family of ${\cal F}_{t}$-measurable $C([-\tau,0];R^{n})$-valued random variables $\xi$ such that $\mathbb{E}\|\xi\|^{q}<\infty$, and by $L^{q}_{{\cal F}_{t}}(R^{n})$ the family of ${\cal F}_{t}$-measurable $R^{n}$-valued random variables $\eta$ such that $\mathbb{E}|\eta|^{q}<\infty$.

Consider the $n$-dimensional hybrid SDE (1.1) on $t\geq 0$, where the coefficients $f:R^{n}\times S\times R_{+}\to R^{n}$ and $g:R^{n}\times S\times R_{+}\to R^{n}\to R^{n\times n}$ are Borel measurable. Assuming SDE (1.1) is unstable, our aim here is to design a Borel measurable control function $u:R^{n}\times S\times R_{+}\to R^{n}$ so that the delay feedback control $u(x(t-\tau)),r(t),t)$ will make the controlled hybrid system (1.4) become stable. Noting that (1.4) is a hybrid stochastic differential delay equation (SDDE), we naturally impose the initial data

[TABLE]

This means at the current time $t=0$ the historical data of the state $\{x(\theta):-\tau\leq u\leq 0\}$ and the mode $r(0)$ are available. For the controlled SDDE (1.4) to have a unique solution on $t\geq 0$ with the initial data (2.1), we impose the global Lipschitz condition (see, e.g., [9, 10, 11, 22]).

Assumption 2.1

There there three positive constants $L_{1},L_{2}$ and $L_{3}$ such that

[TABLE]

for all $(x,y,i,t)\in R^{n}\times R^{n}\times S\times R_{+}$. Moreover,

[TABLE]

for all $(i,t)\in S\times R_{+}$.

We see that this assumption implies the linear growth condition

[TABLE]

for all $(x,i,t)\in R^{n}\times S\times R_{+}$. It is known (see, e.g., [20]) that under Assumption 2.1, the controlled SDDE (1.4)) with the initial data (2.1) has a unique solution $x(t)$ on $t\geq 0$ and the solution has the property that

[TABLE]

where throughout this paper we use the notation $x_{t}=\{x(t+u):-\tau\leq u\leq 0\}$ which is a $C([-\tau,0];R^{n})$-valued stochastic process on $t\geq 0$. To emphasise the role of the initial data at time 0, we will denote the solution by $x(t;\varphi,r_{0},0)$ and the Markov chain by $r(t;r_{0},0)$. Let $t_{0}\geq 0$, $x_{t_{0}}=\{x(t+u;\varphi,r_{0},0):-\tau\leq u\leq 0\}$ and $r(t_{0})=r(t_{0};r_{0},0)$. Moreover, denote the unique solution of the SDDE (1.4) on $t\geq t_{0}$ with the initial data $x_{t_{0}}$ and $r(t_{0})$ at time $t_{0}$ by $x(t;x_{t_{0}},r(t_{0}),t_{0})$ and the corresponding Markov chain by $r(t;r(t_{0}),t_{0})$. We then see the flow property that

[TABLE]

for all $t\geq t_{0}$.

Let us now return to the (non-delay) controlled hybrid SDE (1.2). Instead of $x(t)$ we will use $y(t)$ for the state to distinguish it from the solution of the SDDE (1.4). That is, we consider the auxiliary controlled hybrid SDE

[TABLE]

From now on, we will fix a number $p>0$. It is known (see, e.g., [20]) that under Assumption 2.1, the SDE (2.7) with the initial data $y(t_{0})\in L^{p}_{{\cal F}_{t_{0}}}(R^{n})$ and $r(t_{0})=r_{0}\in M_{{\cal F}_{t_{0}}}(S)$ at time $t_{0}$ has a unique solution $y(t)$ on $t\geq t_{0}$ which has the property that $\mathbb{E}|y(t)|^{p}<\infty$ for all $t\geq t_{0}$. We will denote the solution by $y(t;y(t_{0}),r(t_{0}),t_{0})$. As we mentioned before, there are already many papers devoted to the designation of the control function $u:R^{n}\times S\times R_{+}\to R^{n}$ for this SDE to be exponentially stable in the $p$th moment. We can therefore simply assume the exponential stability of this SDE.

Assumption 2.2

Let $p>0$. Assume that there is a pair of positive constants $M$ and $\gamma$ such that the solution of the auxiliary controlled hybrid SDE (2.7) satisfies

[TABLE]

for all $y(t_{0})\in L^{p}_{{\cal F}_{t_{0}}}(R^{n})$ and $r(t_{0})\in M_{{\cal F}_{t_{0}}}(S)$.

Our aim in this paper is to show that this same control function also makes the delay controlled system (1.4) to be exponentially stable in the $p$th moment as long as $\tau$ is sufficiently small (namely we make state observations frequently enough). To prove this result, let us present a number of lemmas in the next section.

3 Lemmas

In this section, we will fix the initial data (2.1) arbitrarily. We will write the solution $x(t;\varphi,r_{0},0)=x(t)$ of the controlled hybrid SDDE (1.4) with the initial data (2.1) and the Markov chain $r(t;r_{0},0)=r(t)$ on $t\geq 0$. We emphasize that $x_{t}\in L^{p}_{{\cal F}_{t}}(C)$ and $x(t)\in L^{p}_{{\cal F}_{t}}(R^{n})$ (please recall (2.5)). We also emphasize once again that we fix $p>0$ throughout this paper.

Lemma 3.1

Under Assumption, for any $t_{0}\geq 0$ and $T\geq 0$,

[TABLE]

[TABLE]

[TABLE]

where

[TABLE]

in which $C_{p}=[p^{p+1}/2(p-1)^{p-1}]^{p/2}$ for $p\geq 2$. (Of course, $K_{1}$–$K_{3}$ depend on $L_{1}$–$L_{3}$ but we do not want to emphasise them explicitly as $L_{1}$–$L_{3}$ are fixed once the underlying SDE is given but we usually need to choose $p,\tau,T$ to fit into the underlying situation.)

Proof. We fix $t_{0}\geq 0$ and $T\geq 0$ arbitrarily. We first prove the first assertion for $p\geq 2$. By the Itô formula and Assumption 2.1, it is straightforward to show from (1.4) that for $t\in[t_{0},t_{0}+T+\tau]$

[TABLE]

But

[TABLE]

Therefore

[TABLE]

As the last term on the right-hand-side of the inequality above is increasing in $t$, we must have

[TABLE]

An application of the well-known Gronwall inequality yields

[TABLE]

This is the required assertion (3.1) when $p\geq 2$. When $p\in(0,2)$, we can apply the Itô formula to $|x(t)|^{2}$ and then take the conditional expectation given ${\cal F}_{t_{0}}$ to get that

[TABLE]

for all $t\in[t_{0},t_{0}+T+\tau]$. Hence

[TABLE]

Taking the expectation on both sides gives the required assertion (3.1) when $p\in(0,2)$. In other words, we have shown that (3.1) holds for all $p>0$.

Let us proceed to prove the second assertion, namely (3.2) for $p\geq 2$. It is easy to show from (1.4) that

[TABLE]

where $C_{p}$ has been defined in the statement of the lemma. Substituting (3.1) into this yields (3.2) when $p\geq 2$. For $p\in(0,2)$, we note that

[TABLE]

Substituting (3.11) into the above implies

[TABLE]

Then

[TABLE]

Taking the expectation on both sides and recalling the definition of $K_{2}(p,\tau,T)$, we see that the required assertion (3.2) holds for $p\in(0,2)$ as well.

Similarly, we can show the third assertion (3.3). The proof is complete. $\Box$

Lemma 3.2

Let Assumption 2.1 hold. Fix $t_{0}\geq\tau$ and $T\geq 0$ arbitrarily. Write $y(t;x(t_{0}),r(t_{0}),t_{0})=y(t)$ for $t\geq t_{0}$. Then, for $t\in[t_{0},t_{0}+T+\tau]$,

[TABLE]

where

[TABLE]

in which $K_{3}(p,\tau,T)$ has been defined in Lemma 3.1.

Proof. We first show the assertion for $p\geq 2$. By the Itô formula and Assumption 2.1, it is straightforward to show that for $t\in[t_{0},t_{0}+T+\tau]$,

[TABLE]

But

[TABLE]

Hence

[TABLE]

An application of the Gronwall inequality gives the assertion for $p\geq 2$. Let us now consider the case when $p\in(0,2)$. In a similar way as Lemma 3.1 was proved, we can show that

[TABLE]

for $t\in[t_{0},t_{0}+T+\tau]$. We can then show that the assertion holds for $p\in(0,2)$ using the technique of conditional expectation as we did in the proof of Lemma 3.1. The proof is therefore complete.

4 Main Result

We can now form our main theorem in this paper.

Theorem 4.1

Under Assumptions 2.1 and 2.2, there is a positive number $\tau^{*}$ such that the solution of the controlled SDDE (1.4) has the properties that

[TABLE]

and

[TABLE]

as long as $\tau<\tau^{*}$. In other words, the controlled SDDE (1.4) is exponentially stable in the $p$th moment as well as with probability one provided $\tau<\tau^{*}$.

Before the proof, let us make an important remark on how to determine the value of $\tau^{*}$ so that this theorem can be implemented in practice.

Remark 4.2

The use of this theorem in practice depends very much on the value of $\tau^{*}$. We describe a method to determine it. Set $p_{0}=0\vee(p-1)$. Choose a constant $\varepsilon\in(0,1)$ and let

[TABLE]

Let $\tau^{*}$ be the unique positive root to the following equation

[TABLE]

of $\tau$, where both $K_{3}(p,\tau,T)$ and $K_{4}(p,\tau,T)$ have been defined in Section 3. We observe that $\tau^{*}$ exists uniquely and is positive as the left-hand-side term of equation (4.4) is an increasing continuous function of $\tau$ which starts from 0 when $\tau=0$ and tends to infinity as $\tau\to\infty$. However, we do not have the explicit formula for the root $\tau^{*}$ though it can be solved numerically, for example, by MATHEMATICA. We also observe that it is more desirable in practice if we could find a larger value of $\tau^{*}$. Note that once $p,L,M,\gamma$ are given, the root $\tau^{*}$ depends on the choice of $\varepsilon$. That is, $\tau^{*}=\tau^{*}(\varepsilon)$. It would be useful if we could find the optimal $\bar{\varepsilon}\in(0,1)$ in the sense that

[TABLE]

However, this is an open problem.

Proof of Theorem 4.1. To make it clearer, we divide the proof into three steps.

Step 1. We will simply write $K_{3}(p,\tau,T)=K_{3}$ and $K_{4}(p,\tau,T)=K_{4}$. We let $\tau^{*}$ be determined in the way as described in Remark 4.2. Fix $\tau\in(0,\tau^{*})$ and the initial data (2.1). Write $x(t;\varphi,r_{0},0)=x(t)$ and $r(t;r_{0},0)=r(t)$ for $t\geq 0$. Let us first consider $x(t)$ on $t\in[\tau,2\tau+T]$ which can be regarded as the solution of the SDDE (1.4) with initial data $x_{\tau}$ and $r(\tau)$ at time $t=\tau$. Also consider the solution $y(t;x(\tau),r(\tau),\tau)$ of the SDE (2.7) on $t\in[\tau,\tau+T]$ with initial data $x(\tau)$ and $r(\tau)$ at time $t=\tau$. We simply write $y(\tau+T;x(\tau),r(\tau),\tau)=y(\tau+T)$. By Assumption 2.2,

[TABLE]

Moreover, by the elementary inequality $(a+b)^{p}\leq 2^{p_{0}}(a^{p}+b^{p})$ for any $a,b\geq 0$ (please recall that $p_{0}:=0\vee(p-1)$ which has been defined in Remark 4.2), we have

[TABLE]

Using (4.5) and Lemma 3.2, we get

[TABLE]

On the other hand, by Lemma 3.1, we have

[TABLE]

Substituting (4.6) into (4) and noting from (4.3) that

[TABLE]

we get

[TABLE]

But, as $\tau<\tau^{*}$, we see from (4.4) that

[TABLE]

We may therefore write

[TABLE]

for some $\lambda>0$. It then follows from (4.8) that

[TABLE]

Step 2. Let us now consider the solution $x(t)$ on $t\in[2\tau+T,\tau+2(\tau+T)]$. By property (2.6), this can be regarded as the solution of the SDDE (1.4) with the initial data $x_{2\tau+T}$ and $r(2\tau+T)$ at $t=2\tau+T$. In the same way as we did in Step 1, we can show

[TABLE]

This, together with (4.10), implies

[TABLE]

Repeating this procedure, we have

[TABLE]

for all $k=1,2,\cdots$. But, by Lemma 3.1

[TABLE]

This, together with (4.10), yields

[TABLE]

for all $k=0,1,2,\cdots$. Hence, for $t\in[\tau+k(\tau+T),\tau+(k+1)(\tau+T)]$,

[TABLE]

This implies

[TABLE]

In other words, we have shown the required assertion (4.1).

Step 3. It now follows from (4.11) that

[TABLE]

for all $k\geq 1$. By the Borel–Cantelli lemma (see, e.g., [11, Lemma 2.4 on page 7]), we obtain that for almost all $\omega\in\Omega$, there is an integer $k_{0}=k_{0}(\omega)$ such that

[TABLE]

This implies easily that

[TABLE]

for almost all $\omega\in\Omega$. The other assertion (4.2) must therefore hold. The proof is hence complete.

5 Implementation

In this section we will demonstrate how to implement our theory in order to stabilise the given unstable hybrid SDE (1.1) by a delay feedback control $u(x(t-\tau),r(t),t)$ in the drift. Our new Theorem 4.1 enables us to transfer the stabilisation problem of (1.4) to the classical stabilisation problem of (2.7) where the feedback control $u(x(t),r(t),t)$ is of no-delay. The use of our new Theorem 4.1 depends on the design of the control function $u(x,i,t)$ that makes the controlled SDE (2.7) become exponentially stable in the $p$th moment as described in Assumption 2.2. There are lots of known criteria on the $p$th moment exponential stability of hybrid SDEs (see, e.g., [12, 19, 28, 29]), which can be applied to design the control function. What we are going to demonstrate here is to apply [20, Theorem 5.8 on page 166] to establish a criterion for the control function to satisfy. For this purpose, we will impose a new assumption.

Assumption 5.1

Let $p>0$. Assume that there are real numbers $\alpha_{i}$, $i\in S$, such that

[TABLE]

for all $(x,i,t)\in(R^{n}-\{0\})\times S\times R_{+}$ while

[TABLE]

is a non-singular M-matrix.

The following theorem shows that if the control function $u(x,t,i)$ makes Assumption 5.1 to hold, then the controlled SDE (2.7) is exponentially stable in the $p$th moment.

Theorem 5.2

Let Assumptions 2.1 and 5.1 hold. Then Assumption 2.2 holds with

[TABLE]

where

[TABLE]

Consequently, Theorem 4.1 holds under Assumptions 2.1 and 5.1.

Proof. We observe that all $\theta_{i}$’s are positive as ${\cal A}$ is a nonsingular M-matrix (see, e.g., [20]). It follows from (5.4) that

[TABLE]

We will apply [20, Theorem 5.8 on page 166] to prove this theorem. We first consider the controlled SDE (2.7) on $t\geq t_{0}$ in the case where the initial data are deterministic, namely $y(t_{0})=y_{0}\in R^{n}$ and $r(t_{0})=r_{0}\in S$ at time $t_{0}(\geq 0)$. We will write the solution $y(t;y_{0},r_{0},t_{0})=x(t)$ and the Markov chain $r(t;r_{0},t_{0})=r(t)$. Clearly, the assertion holds if $y_{0}=0$ so we need to consider $y_{0}\not=0$. In this case, $x(t)\not=0$ a.s. for all $t\geq 0$ (see, e.g., [20, Lemma 5.1 on page 164]). Define the Lyapunov function $V:R^{n}-\{0\})\times S\times R_{+}$ by $V(x,i,t)=\theta_{i}|x|^{p}$. So

[TABLE]

where both $\beta_{1}$ and $\beta_{2}$ have been defined in the statement of the theorem. Moreover, the generalised Itô formula (see, e.g., [20, Theorem 1.45 on page 48]) shows

[TABLE]

where $M(t)$ is a local Martingale on $t\geq 0$ (but its explicit form is of no use here) and

[TABLE]

for $(x,i,t)\in R^{n}-\{0\})\times S\times R_{+}$. By (5.1) and (5.5), we then have

[TABLE]

An application of [20, Theorem 5.8 on page 166] yields

[TABLE]

where both $M$ and $\gamma$ have been defined in the statement of Theorem 5.2. We now consider the general case, namely the controlled SDE (2.7) on $t\geq t_{0}$ with the initial data $y(t_{0})=y_{0}\in L^{p}_{{\cal F}_{t_{0}}}(R^{n})$ and $r(t_{0})=r_{0}\in M_{{\cal F}_{t_{0}}}(S)$ at time $t_{0}$. In this case, by the technique of conditional expectation, we derive

[TABLE]

for $t\geq t_{0}$. In other words, we have shown that Assumption 2.2 holds. Consequently, Theorem 4.1 holds under Assumptions 2.1 and 5.1. The proof is complete.

Accordingly, we can implement our theory in two steps assuming that the coefficients $f$ and $g$ of the given hybrid SDE (1.1) satisfy Assumption 2.1:

• Step 1

Design the control function $u(x,i,t)$ which satisfies Assumptions 2.1 and 5.1. Compute $\theta_{i}$’s by (5.4) and determine both $M$ and $\gamma$ by (5.3).

• Step 2

Choose a constant $\varepsilon\in(0,1)$ and compute $T$ by (4.3). Find the unique positive root $\tau^{*}$ of equation (4.4) numerically. Make sure the delay feedback control $u(x(t-\tau),r(t),t)$ acts quickly enough in the sense $\tau<\tau^{*}$. Then the controlled hybrid SDDE (1.4) is exponentially stable in the $p$th moment as well as in probability one.

Let us discuss an example to illustrate our theory.

Example 5.3

Consider a second order SDE

[TABLE]

where $\dot{B}(t)$ is a scalar white noise (informally thought as the derivative of a scalar Brownian motion $B(t)$), $r(t)$ is a Markov chain taking values in the state space $S=\{1,2\}$ with the generator

[TABLE]

and the coefficients are specified by

[TABLE]

This SDE has been used to describe, for example, the nonlinear hybrid stochastic oscillator (see, e.g., [11]). Introducing $x(t)=(x_{1}(t),x_{2}(t))^{T}=(z(t),\dot{z}(t))^{T}$, we can write the oscillator as the two-dimensional hybrid SDE

[TABLE]

where

[TABLE]

The computer simulation (see Figure 5.1) shows this given hybrid SDE is unstable.

Let us now apply our new theory to design a delay feedback control to stabilise the SDE. To show our theory can be applied to cope with various practical situations, we consider a structure feedback control in this example. Due to the page limit, we only discuss an interesting situation, where

• •

the state, in both modes, could only be observed in $x_{1}$-component and the control could only be fed into $x_{1}$-component too.

For example, this is the case when $x_{1}$ represents the velocity and $x_{2}$ the distance in a hybrid stochastic oscillation system while only the velocity is observable and controllable.

To make it simple, we will only seek for a linear control function. In terms of mathematics, our control function has the form

[TABLE]

where $d_{1}$ and $d_{2}$ are both positive numbers to be chosen. It is straightforward to show that Assumption 2.1 is satisfied with

[TABLE]

It is also easy to show

[TABLE]

for $(x,i)\in R^{2}-\{0\})\times S$, where

[TABLE]

namely

[TABLE]

In this example, we aim to stabilise the SDE in the sense of almost sure exponential stability so it is wise to choose $p\in(0,1)$ to make use of the stabilisation effect of the Brownian motion (see, e.g., [19]). We choose $p=0.99$ and $d_{1}$ for $0.1-d_{1}=-0.464+0.08p$, namely $d_{1}=0.4848$, while $d_{2}$ for $0.28125-0.5d_{2}=-0.125(1-p)$, namely $d_{2}=0.5650$. Consequently

[TABLE]

It is then easy to see that $\alpha_{1}$ and $\alpha_{2}$ in Assumption 5.1 are: $\alpha_{1}=0.3848$ and $\alpha_{2}=0.0012$. The matrix defined by (5.2) becomes

[TABLE]

which is a nonsingular M-matrix. In other words, we have verified Assumption 5.1 for $p=0.99$. We can then determine $\theta_{1}=3.891286$, $\theta_{2}=4.388653$ and, hence, $M=1.127816$ and $\gamma=0.2278604$ by (5.4) and (5.3), respectively. By Theorem 5.2, the controlled SDE

[TABLE]

with the control function $u(x,i)$ defined by (5.7) is almost surely exponentially stable (please note that the $p$th moment exponential stability implies the almost sure exponential stability [11]). The computer simulation (see Figure 5.2) supports this theoretical result clearly.

However, our aim is to use the delay feedback control. For this purpose, we further choose $\varepsilon=0.94$ and compute $T=0.7994283$ by (4.3). Equation (4.4) becomes

[TABLE]

which has the unique positive root $\tau^{*}=2.93\times 10^{-6}$ (that is about 92 seconds if the time unit is of year). By Theorem 5.2, the delay-feedback controlled system

[TABLE]

with the control function $u(x,i)$ defined by (5.7) is almost surely exponentially stable as long as $\tau<2.93\times 10^{-6}$. Once again, the computer simulation (see Figure 5.3) supports this theoretical result clearly.

Remark 5.4

Actually, the $\tau^{*}$ obtained in this example is optimal, to a certain degree according to our theory. More precisely, given that all the coefficients and $\Gamma$ in (5.6) are known, the largest $\tau^{*}$ is found by a numerical programme which searches for $p\in(0,1)$ and $\epsilon\in(0,1)$ to maximise $\tau^{*}$ according to Theorem 5.2.

6 Conclusion

For some unstable hybrid stochastic differential equations, it is much harder to design the feedback control based on past states than current states. However, the feedback control based on past states are more practical than that based on current states. In this paper, we proposed a new theorem to connect the delay feedback control to the feedback control without delays. Such a result makes it possible to construct the delay feedback control $u(x(t-\tau),r(t),t)$, given that the feedback control $u(x(t),r(t),t)$ is known. Therefore, various existing results on the classical stabilisation problem together with the novel theorem proved in this paper enable us to design the delay feedback control much more easily.

Numerical simulations were provided to demonstrate the theoretical results as well as the way to find the lower bound of the length of the time delay. In addition, the optimal lower bound (according to our theory) was found numerically for a given unstable hybrid stochastic differential equation and a given format of the control function.

Appendix

Consider a scalar SDE

[TABLE]

on $t\geq 0$, where $B(t)$ is a scalar Brownian. It is known (see, e.g., [11]) that for any given initial value $x(0)=x_{0}\in R$, there is a unique global solution to the SDE but the equation is not exponentially stable in the $p$th moment for any $p\geq 2$.

We design a (non-delay) feedback control $-2x^{3}(t)$ in the drift so that the controlled SDE is

[TABLE]

It is almost straightforward to show that for any initial value $x(0)=x_{0}\in R$, the solution of this controlled SDE satisfies

[TABLE]

In other words, the feedback control stabilises the given system (6.1) in the sense of the 4th moment exponential stability.

Let us now show that for any $\varepsilon>0$ (no matter how small it is), the corresponding delay feedback control $-2x^{3}(t-\varepsilon)$ in the drift can NOT stabilise the given system (6.1) in the sense of the 4th moment exponential stability. In other words, we will show that the corresponding delay feedback controlled system

[TABLE]

is NOT exponentially stable in the 4th moment. It is easy to see that this SDDE has a unique global solution $x(t)$ for any initial data $\{x(\theta):-\varepsilon\leq\theta\leq 0\}=\varphi\in C([-\varepsilon,0];R)$. If this SDDE were exponentially stable in the 4th moment, then for any initial data $=\varphi\in C([-\varepsilon,0];R)$, the 4th moment $\mathbb{E}|x(t)|^{4}$ of the solution must be finite and tend to 0 exponentially fast as $t\to\infty$. We consider a special initial data $\varphi\in C([-\varepsilon,0];R)$ such that

[TABLE]

where $\bar{z}$ is the unique positive root to the following equation on $z\geq 2$

[TABLE]

As we indicated that $\mathbb{E}|x(t)|^{4}<\infty$ for all $t\geq 0$, we can show easily that, for $t\in[0,0.5\varepsilon]$,

[TABLE]

It is easy to see that $\mathbb{E}|x(t)|^{2}\geq\bar{z}^{2}>4$ for $t\in[0,0.5\varepsilon]$. By the Hölder inequality,

[TABLE]

for $s\in[0,0.5\varepsilon]$. It then follows from (Appendix) that

[TABLE]

for $t\in[0,0.5\varepsilon]$. By the well-known comparison theorem, $\mathbb{E}|x(t)|^{2}\geq u(t)$ for $t\in[0,0.5\varepsilon]$, where

[TABLE]

It is known (see, e.g., [11, 20]) that equation (6.9) has its explicit solution

[TABLE]

on $t\in[0,0.5\varepsilon)$. This implies $u(t)\to\infty$ as $t\to 0.5\varepsilon$. Consequently, $\mathbb{E}|x(t)|^{2}$ and, hence $\mathbb{E}|x(t)|^{4}\to\infty$ as $t\to 0.5\varepsilon$. This is in contradiction to that $\mathbb{E}|x(t)|^{4}$ is finite and tends to 0 as $t\to\infty$. We can therefore conclude that the controlled SDDE (6.4) is NOT exponentially stable in the 4th moment no matter how small the $\varepsilon$ is, although its corresponding controlled SDE (6.2) is exponentially stable in the 4th moment.

Acknowledgements

Junhao Hu would like to thank the Natural Science Foundation of China (61876192) for the financial support.

Wei Liu would like to thank the Natural Science Foundation of China (11701378, 11871343), Chenguang Program supported by both Shanghai Education Development Foundation and Shanghai Municipal Education Commission (16CG50), and Shanghai Gaofeng & Gaoyuan Project for University Academic Program Development for their financial support.

Feiqi Deng would like to thank the Natural Science Foundation of China (61503142, 61573156, 61733008) for their financial support.

Xuerong Mao would like to thank the Royal Society (WM160014, Royal Society Wolfson Research Merit Award), the Royal Society and the Newton Fund (NA160317, Royal Society-Newton Advanced Fellowship), and the EPSRC (EP/K503174/1) for their financial support.