Equivalent Stability Notions, Lyapunov Inequality, and Its Application in Discrete-Time Linear Systems with Stochastic Dynamics Determined by an i.i.d. Process

TL;DR

This paper establishes the equivalence of different stability notions for discrete-time linear systems with i.i.d. stochastic dynamics, derives a Lyapunov inequality condition for stability, and demonstrates its application in stability analysis and synthesis.

Contribution

It proves the equivalence of stability notions under i.i.d. assumptions and derives a Lyapunov inequality condition that can be solved as a linear matrix inequality.

Findings

Equivalence of stability notions under i.i.d. stochastic dynamics.

Lyapunov inequality condition for stability in necessary and sufficient form.

Numerical examples demonstrating the approach's effectiveness.

Abstract

This paper is concerned with stability analysis and synthesis for discrete-time linear systems with stochastic dynamics. Equivalence is first proved for three stability notions under some key assumptions on the randomness behind the systems. In particular, we use the assumption that the stochastic process determining the system dynamics is independent and identically distributed (i.i.d.) with respect to the discrete time. Then, a Lyapunov inequality condition is derived for stability in a necessary and sufficient sense. Although our Lyapunov inequality will involve decision variables contained in the expectation operation, an idea is provided to solve it as a standard linear matrix inequality; the idea also plays an important role in state feedback synthesis based on the Lyapunov inequality. Motivating numerical examples are further discussed as an application of our approach.

Equations

$$x_{k+1}=A({\xi }_k)x_k,$$

$$E[A_{ij}({\xi }_k{)}^2]<\infty (\forall i,j=1,\dots ,n),$$

$$\Vert x_0{\Vert }^2\le \delta (ϵ)\Rightarrow E[\Vert x_k{\Vert }^2]\le ϵ(\forall k\in {\mathbf{N}}_0).$$

$$E[\Vert x_k{\Vert }^2]\to 0\mathrm{as}k\to \infty (\forall x_0\in {\mathbf{R}}^n).$$

$$\sqrt{E[||x_k|{|}^2]}\le a||x_0||{\lambda }^k(\forall k\in {\mathbf{N}}_0,\forall x_0\in {\mathbf{R}}^n).$$

$$E[x_{k+1}^{T}P{x}_{k+1}]\le {\lambda }^{2}E[x_k^{T}P{x}_k](\forall k\in {\mathbf{N}}_0,\forall x_0\in {\mathbf{R}}^n).$$

$$E[\Vert x_k{\Vert }^2]\le a^2\Vert x_0{\Vert }^2(\forall k\in {\mathbf{N}}_0).$$

$$x_0=\beta \sum _{i=1}^n{a}_i{\sigma }_i{e}^{(i)}$$

$$\Vert x_0{\Vert }^2$$

$$x_k=\beta \sum _{i=1}^n{a}_i{\sigma }_i{x}_k^{(i)},$$

$$E[\Vert x_k^{(i)}{\Vert }^2]\le 1/(2n^2)(i=1,\dots ,n;\forall k\ge K).$$

$$E[\Vert x_k{\Vert }^2]$$

$$\le {\beta }^{2}E\left[\sum _{i=1}^n{a}_i\Vert {\sigma }_i{x}_k^{(i)}{\Vert }^2\right]$$

$$={\beta }^2\sum _{i=1}^n{a}_{i}E[\Vert x_k^{(i)}{\Vert }^2]$$

$$\le {\beta }^2/(2n)(\forall k\ge K),$$

$$E[\Vert x_K{\Vert }^2]\le \Vert x_0{\Vert }^2/2$$

$$E[\Vert x_{k+K}{\Vert }^2]\le E[\Vert x_k{\Vert }^2]/2(\forall k\in {\mathbf{N}}_0).$$

$$E[\Vert x_k{\Vert }^2]$$

$$\le E[\Vert x_c{\Vert }^2]/2^j$$

$$=2^{c/K}E[\Vert x_c{\Vert }^2](2^{-1/K}{)}^k$$

$$\le 2E[\Vert x_c{\Vert }^2](2^{-1/K}{)}^k(\forall k\in {\mathbf{N}}_0).$$

$$E[\Vert x_c{\Vert }^2]\le {\alpha }_K\Vert x_0{\Vert }^2(\forall c\in [0,K)).$$

$$E[\Vert x_k{\Vert }^2]$$

$$E[x_k^{T}P{x}_k]\le {\lambda }^{2k}{x}_0^{T}P{x}_0(\forall k\in {\mathbf{N}}_0,\forall x_0\in {\mathbf{R}}^n).$$

$${\sigma }_{\min }(P)E[\Vert x_k{\Vert }^2]\le E[x_k^{T}P{x}_k],$$

$${\lambda }^{2k}{x}_0^{T}P{x}_0\le {\sigma }_{\max }(P)\Vert x_0{\Vert }^2{\lambda }^{2k}.$$

$${\Gamma }_{k_1}^{k_2}:=\{\begin{array}{ll}I& (k_2=k_1-1)\\ (A({\xi }_{k_2})/{\lambda }_{ϵ})\dots (A({\xi }_{k_1})/{\lambda }_{ϵ})& (k_2\ge k_1)\end{array}$$

$$x_0^{T}E[({\Gamma }_0^{k-1}{)}^T{\Gamma }_0^{k-1}]x_0\le x_0^T(a^2({\lambda }^2/{\lambda }_{ϵ}^2{)}^{k}I)x_0$$

$$(\forall k\in {\mathbf{N}}_0,\forall x_0\in {\mathbf{R}}^n),$$

$$E[({\Gamma }_{k_1}^{k_2}{)}^T{\Gamma }_{k_1}^{k_2}]\le a^2({\lambda }^2/{\lambda }_{ϵ}^2{)}^{k_2-k_1+1}I$$

$$(\forall k_1,k_2\in {\mathbf{N}}_{0}s.t.k_2\ge k_1).$$

$$P_k^K:={\lambda }_{ϵ}^{-2}I+{\lambda }_{ϵ}^{-2}({\Gamma }_k^k{)}^T{\Gamma }_k^k+\dots +{\lambda }_{ϵ}^{-2}({\Gamma }_k^K{)}^T{\Gamma }_k^K$$

$${\lambda }_{ϵ}^2{P}_k^K-A({\xi }_k{)}^T{P}_{k+1}^{K}A({\xi }_k)=I$$

$$(\forall k,K\in {\mathbf{N}}_{0}s.t.K>k\ge 0),$$

$${\lambda }_{ϵ}^{2}E[x_{k}E[P_k^K]x_k]-E[x_{k+1}^{T}E[P_{k+1}^K]x_{k+1}^T]\ge 0.$$

Full text

I Introduction

Recent advances in computer power are leading to demands for extending the frontier of control technologies to cover wider classes of systems. Stimulated by such a trend, this paper focuses on the class of discrete-time linear systems whose state transition is determined only randomly, also called discrete-time linear random dynamical systems in the field of analytical dynamics [1]. Randomness is fairly common in various kinds of phenomena (e.g., packet interarrival times in networked systems [2] and failure occurrences in distributed systems [3]), and discarding the information about it in modeling might lead to the situation where the controllers designed with the resulting models do not achieve the expected performance for the original objectives. Hence, if the randomness behind the real objects is essential, it should also be modeled and exploited in controller synthesis.

Behaviors and properties of random dynamical systems have been extensively studied, e.g., in [4, 5, 6]. However, studies dealing with such systems in control problems are still rare. Our ultimate goal is to develop a versatile practical framework for controlling such systems by restricting our attention to the linear case. In particular, we aim at developing a systematic analysis and synthesis approach based on linear matrix inequality (LMI) conditions, as in the existing studies on deterministic linear systems [7, 8, 9]. As a step toward such a goal, this paper first shows the equivalence of some stability notions and derives the Lyapunov inequality condition for stability of discrete-time linear random dynamical systems under some key assumptions. Our Lyapunov inequality characterizes stability of such systems in a necessary and sufficient sense and will be a basis for further advanced analysis and synthesis using LMIs.

The state transition of our discrete-time linear random dynamical systems can be seen as determined by an underlying stochastic process, and we assume in this paper that the process is independent and identically distributed (i.i.d.) with respect to the discrete time (hence the process naturally becomes stationary and ergodic [10]); this assumption will play a key role in showing the above stability equivalence. This system class contains various kinds of linear stochastic systems studied in the literature. For example, systems with state-multiplicative noise [7, 11] and switched systems [12] with i.i.d. switching signals (which correspond to Markov jump systems [13] with transition probability uniform in all current modes) belong to this class. Hence, the results in this paper can be seen as a generalization of those for such particular systems, and would become a sort of center point for bridging and unifying the associated existing results.

For reference, we briefly summarize the technical aspect of the contributions in this paper through the comparison with the closely related earlier studies [14, 15]. In [14], a necessary and sufficient stability condition is shown for discrete-time linear systems with stochastic dynamics determined by a stationary Markov process. Since i.i.d. processes are a special case of stationary Markov processes, one might consider that our results could be covered by those for Markov jump systems. However, this is not true because the above earlier results are derived with the assumption that the maximum singular value of the random coefficient matrix (depending on the Markov process) is essentially bounded, which makes it impossible to deal with random coefficient matrices involving, e.g., normally distributed elements. This paper will use a milder assumption for this part (see Assumption 2 introduced later). Hence, our results cannot generally be covered by those in [14] (and vice versa). In the earlier study [15] of the authors, a sufficient stability condition was shown as a part of the contributions for essentially the same stochastic systems in the present paper. However, only exponential stability was dealt with and the necessity assertion of the condition was not discussed even for that stability notion. This paper will complement this earlier study from several viewpoints.

The contents of this paper are as follows. In Section II, the stochastic system to be dealt with in this paper is described, and three stability notions are introduced: asymptotic stability, exponential stability [16] and quadratic stability. Then, the equivalence of those stability notions is proved in Section III. Then, the Lyapunov inequality is derived in Section IV as a necessary and sufficient condition for quadratic stability. Since our Lyapunov inequality will involve decision variables contained in the operation of expectation, we will also provide ideas for solving the inequality as a standard LMI involving no expectation operation. The stabilization feedback synthesis based on the Lyapunov inequality is further discussed in Section V. Finally, two numerical examples are provided for our stability analysis and synthesis in Section VI. The first example has the role of demonstrating that the Lyapunov inequality indeed gives a necessary and sufficient condition not only for quadratic stability but for exponential stability, as is theoretically indicated. On the other hand, the second example is provided for showing the potential of our approach by tackling a challenging problem; more specifically, we consider stabilizing the discrete-time system obtained through discretizing a continuous-time deterministic linear system with a randomly time-varying sampling interval, which is inspired by the studies on aperiodic sampling [17, 18] (related, e.g., to packet interarrival times in networked systems).

We use the following notation in this paper. ${\bf R}$, ${\bf R}_{+}$ and ${\bf N}_{0}$ denote the set of real numbers, that of positive real numbers and that of non-negative integers, respectively. ${\bf R}^{n}$ and ${\bf R}^{m\times n}$ denote the set of $n$-dimensional real column vectors and that of $m\times n$ real matrices, respectively. ${\bf S}^{n\times n}$ and ${\bf S}^{n\times n}_{+}$ denote the set of $n\times n$ symmetric matrices and that of $n\times n$ positive definite matrices, respectively. $\sigma_{\rm max}(\cdot)$ and $\sigma_{\rm min}(\cdot)$ denote the maximum and minimum singular values of the matrix $(\cdot)$, respectively. $||(\cdot)||$ denotes the Euclidean norm of the vector $(\cdot)$. ${\rm row}(\cdot)$ denotes the vectorization of the matrix $(\cdot)$ in the row direction, i.e., ${\rm row}(\cdot)=[{\rm row}_{1}(\cdot),\ldots,{\rm row}_{m}(\cdot)]$ where $m$ is the number of rows of the matrix and ${\rm row}_{i}(\cdot)$ denotes the $i$th row. $\otimes$ denotes the Kronecker product. ${\rm diag}(\cdot)$ denotes the (block-)diagonal matrix. $E[(\cdot)]$ denotes the expectation of the random variable $(\cdot)$; this notation is also used for the expectation of the random matrix $(\cdot)$. If $s$ is a random variable obeying the distribution ${D}$, then we represent it as $s\sim{D}$.

II Stability of Discrete-Time Linear Systems with Stochastic

Dynamics

II-A *Discrete-Time Linear Systems with Stochastic

Dynamics*

Let us consider the $Z$-dimensional discrete-time stochastic process $\xi$, which is the sequence of $Z$-dimensional random vectors $\xi_{k}$ with respect to the discrete time $k\in{\bf N}_{0}$, and make the following key assumption on it.

Assumption 1

$\xi_{k}$* is independent and identically distributed (i.i.d.) with respect to $k\in{\bf N}_{0}$.*

This assumption naturally makes $\xi$ stationary and ergodic [10]. For this stochastic process $\xi$, we denote the cumulative distribution function of $\xi_{k}$ and the corresponding support by ${\cal F}(\xi_{k})$ and ${\boldsymbol{\mit\Xi}}$, respectively. By definition, ${\boldsymbol{\mit\Xi}}\subset{\bf R}^{Z}$, and ${\boldsymbol{\mit\Xi}}$ corresponds to the set of values that $\xi_{k}$ can take.

Let us further consider the discrete-time linear system

[TABLE]

where $x_{k}\in{\bf R}^{n}$, $A:{\boldsymbol{\mit\Xi}}\rightarrow{\bf R}^{n\times n}$, and the initial state $x_{0}$ is assumed to be deterministic. Since $A(\xi_{k})$ is a random matrix (while $A(\cdot)$ itself is a deterministic mapping), the dynamics of the above system is stochastic. To ensure mathematical rigor throughout this paper, we make the following assumption on the coefficient matrix $A(\xi_{k})$ of the system.

Assumption 2

The squares of elements of $A(\xi_{k})$ are all Lebesgue integrable, i.e.,

[TABLE]

where $A_{ij}(\xi_{k})$ denotes the $(i,j)$-entry of $A(\xi_{k})$.

In this paper, we say that the expectation of a random variable is well-defined, if the random variable is Lebesgue integrable; hence, $E[A_{ij}(\xi_{k})^{2}]$ satisfying (2) is said to be well-defined. This term is also used for the expectation of a random matrix when its elements are all Lebesgue integrable.

The aim of this paper is to develop a theoretical basis of stability analysis and synthesis for system (1) with $\xi$ satisfying Assumptions 1 and 2. Since we have introduced no essential restrictions on ${\cal F}(\cdot)$ and $A(\cdot)$, this system covers a wide class of discrete-time linear systems with stochastic dynamics; indeed, system (1) is the most general for representing the discrete-time linear finite-dimensional systems with stochastic dynamics (without additive inputs) under Assumptions 1 and 2. Assumption 2 would not become a problem from the practical viewpoint, and hence, the only essential restriction on the system is Assumption 1, which plays a crucial role throughout this paper.

II-B Stability Notions

We next introduce three stability notions for system (1) with $\xi$ satisfying Assumptions 1 and 2. The first and second notions are asymptotic stability and exponential stability [16] defined as follows.

Definition 1** (Asymptotic Stability)**

The system (1) with $\xi$ satisfying Assumptions 1 and 2 is said to be stable in the second moment if for each positive $\epsilon$, there exists $\delta=\delta(\epsilon)$ such that

[TABLE]

In addition, the system is said to be asymptotically stable in the second moment if the system is stable in the second moment and

[TABLE]

Definition 2** (Exponential Stability)**

The system (1) with $\xi$ satisfying Assumptions 1 and 2 is said to be exponentially stable in the second moment if there exist $a\in{\bf R}_{+}$ and $\lambda\in(0,1)$ such that

[TABLE]

The second-moment asymptotic (resp. exponential) stability defined above is also called asymptotic (resp. exponential) mean square stability [16], and is widely used in the field of stochastic systems control. In Definition 2, $\lambda$ is an upper bound of the convergence rate with respect to the sequence $\left(\sqrt{E[||x_{k}||^{2}]}\right)_{k\in{\bf N}_{0}}$.

Compared to the above two notions, the following third notion might not be major in the field of stochastic systems control but is closely related to our main arguments.

Definition 3** (Quadratic Stability)**

The system (1) with $\xi$ satisfying Assumptions 1 and 2 is said to be quadratically stable if there exist $P\in{\bf S}^{n\times n}_{+}$ and $\lambda\in(0,1)$ such that

[TABLE]

In the above definition of quadratic stability, $V(x_{k})=E[x_{k}^{T}Px_{k}]$ is the quadratic Lyapunov function described with the Lyapunov matrix $P$, and (6) requires the existence of a Lyapunov function (i.e., $P$) that decays no slower than the rate $\lambda^{2}\ (<1)$, as is the case with deterministic systems [7].

Here, to ensure mathematical rigor, we show that Assumptions 1 and 2 lead to the well-definedness of the expectations referred to in the above definitions. As a step for this end, we first note two facts. The first fact is that if $s_{1}\leq s_{2}$ (resp. $s_{1}<s_{2}$) for each sample of the pair of the two random variables $s_{1}$ and $s_{2}$, then $E[s_{1}]\leq E[s_{2}]$ (resp. $E[s_{1}]<E[s_{2}]$); since this fact is almost trivial, we use it throughout this paper without any specific notes. Compared to the first fact, the second fact might not be trivial and we would like to summarize it as in the following lemma, which can be shown with the Cauchy-Schwarz inequality.

Lemma 1

If the expectations of $s_{1}^{2}$ and $s_{2}^{2}$ are well-defined for the random variables $s_{1}$ and $s_{2}$, then the expectation of $s_{1}s_{2}$ also is.

Then, by using the above two facts, we can obtain the following result: for the random vector $s_{1}$ and the square random matrix $S_{2}$ (of the compatible size) such that $s_{1}$ and $S_{2}$ are independent of each other and $E[S_{2}]$ is well-defined, the expectation $E[s_{1}^{T}S_{2}s_{1}]\ (=E[s_{1}^{T}E[S_{2}]s_{1}])$ is well-defined if $E[\|s_{1}\|^{2}]$ is. Hence, by taking $s_{1}=x_{k}$ and $S_{2}=A(\xi_{k})^{T}A(\xi_{k})$ under Assumptions 1 and 2, we can show that if $E[\|x_{k}\|^{2}]$ is well-defined, then $E[\|x_{k+1}\|^{2}]$ also is; the well-definedness of $E[S_{2}]=E[A(\xi_{k})^{T}A(\xi_{k})]$ can be ensured by Lemma 1 under Assumption 2. A recursive use of this result leads to the well-definedness of $E[\|x_{k}\|^{2}]$ for every $k\in{\bf N}_{0}$. The well-definedness of $E[x_{k}^{T}Px_{k}]$ can be ensured in a similar fashion. Hence, the expectations in Definitions 1 through 3 are all well-defined under Assumptions 1 and 2.

III Equivalence of Three Stability Notions

Three stability notions were introduced in the preceding section: asymptotic stability, exponential stability and quadratic stability. Since quadratic stability is usually introduced as a notion related with deterministic time-invariant Lyapunov matrices (as in Definition 3), it is not equivalent to asymptotic stability and exponential stability, in general. For example, in the case of deterministic linear time-varying systems, such equivalence is known to fail [19]. Hence, one might be concerned about the possibility of a similar situation for the system (1) since it can be viewed as a deterministic linear time-varying system when we discard the information about underlying randomness. However, we can actually establish equivalence of all these notions also for the stochastic system (1) (and the present stability definitions), provided that Assumptions 1 and 2 are satisfied, as in the case with deterministic linear time-invariant (LTI) systems. Showing this non-trivial equivalence is one of the main results in this paper.

III-A Equivalence between Asymptotic Stability and Exponential Stability

We first give the proof of the following theorem about equivalence between asymptotic stability and exponential stability (similar equivalence is known to hold for deterministic linear systems [20]).

Theorem 1

Suppose $\xi$ satisfies Assumption 1 and $A(\xi_{k})$ satisfies Assumption 2. The following two conditions are equivalent.

The system (1) is asymptotically stable in the second moment. 2. 2.

The system (1) is exponentially stable in the second moment.

Proof:

2$\Rightarrow$1: It follows from (5) and $0<\lambda<1$ that

[TABLE]

This leads us to (3) with $\delta(\epsilon)=\epsilon/a^{2}$, which means the second-moment stability of the system. In addition, (4) readily follows from (5) since $0<\lambda<1$. Hence, by definition, the system is asymptotically stable in the second moment.

1$\Rightarrow$2: Linearity of the system (1) frequently used in this part of the proof is not explicitly referred to so as not to make the arguments verbose. We first introduce the decomposition

[TABLE]

with the scalars $\beta$, $a_{i}\geq 0\ (i=1,\ldots,n)$ satisfying $\sum_{i=1}^{n}a_{i}=1$, the integers $\sigma_{i}\in\{-1,1\}\ (i=1,\ldots,n)$ and the standard basis vectors $e^{(i)}\ (i=1,\ldots,n)$ for the $n$-dimensional Euclidean space. By definition, we have

[TABLE]

Associated with this decomposition of $x_{0}$, we can also decompose the corresponding $x_{k}$ as

[TABLE]

where $x^{(i)}_{k}$ is the state at $k$ for the initial state $x_{0}=e^{(i)}$. It follows from (4) that there exists $K\in{\bf N}_{0}$ such that

[TABLE]

Then, we have

[TABLE]

where the first inequality follows from Jensen’s inequality. Hence, it follows from (9) and (12) that

[TABLE]

for the same $K$. Since this inequality holds regardless of $x_{0}\in{\bf R}^{n}$ and since $\xi$ satisfies Assumption 1 (in particular, the stationarity assumption of $\xi$), we further have

[TABLE]

For each $k\in{\bf N}_{0}$, take $j$ and $c$ such that $k=c+jK\ (0\leq c<K)$. Then, a recursive use of (14) leads to

[TABLE]

Since Assumptions 1 and 2 ensure that $E[\|x_{c}\|^{2}]$ is well-defined for every $c\in[0,K)$ (from the arguments in the preceding subsection), there exists a bounded positive scalar $\alpha_{K}$ such that

[TABLE]

This, together with (15), leads us to

[TABLE]

which implies the existence of $a=2\alpha_{K}$ and $\lambda=2^{-1/K}$ such that $a>0$, $0<\lambda<1$ and (5) hold. Hence, by definition, the system is exponentially stable in the second moment. This completes the proof. ∎

Note that the above proof actually showed that the system is exponentially stable if and only if (4) holds; in other words, (3) was not used in the part “1$\Rightarrow$2”. This readily leads us to the following corollary as an implicit result about asymptotic stability.

Corollary 1

Suppose $\xi$ satisfies Assumption 1 and $A(\xi_{k})$ satisfies Assumption 2. The system (1) is asymptotically stable in the second moment if and only if (4) holds.

The property described with (4) is called attractivity [20]. Although asymptotic stability is usually defined not only with attractivity but also with stability (as in Definition 1), it is known in the deterministic systems case that asymptotic stability can be ensured only with attractivity if the system is linear and time-invariant. Hence, the above corollary corresponds to a stochastic counterpart of this conventional result because of Assumption 1 (although system (1) itself is not time-invariant).

III-B *Equivalence between Exponential Stability and Quadratic

Stability*

The remaining issue in this section is to show the following theorem.

Theorem 2

Suppose $\xi$ satisfies Assumption 1 and $A(\xi_{k})$ satisfies Assumption 2. The following two conditions are equivalent.

The system (1) is exponentially stable in the second moment. 2. 2.

The system (1) is quadratically stable.

Proof:

2$\Rightarrow$1: A recursive use of (6) leads to

[TABLE]

For the left-hand side of this inequality,

[TABLE]

while for the right-hand side,

[TABLE]

Hence, we have (5) with $a=\sqrt{\sigma_{\max}(P)/\sigma_{\min}(P)}$ and the same $\lambda$, which means by definition that the system is exponentially stable in the second moment.

1$\Rightarrow$2: Take a positive $\epsilon$ such that $\lambda_{\epsilon}:=\lambda+\epsilon<1$ and define

[TABLE]

for non-negative integers $k_{1}$ and $k_{2}\ (\geq k_{1}-1)$. Then, (5) can be rewritten as

[TABLE]

where the well-definedness of the expectation in the left-hand side is ensured under Assumptions 1 and 2 (in essentially the same manner as Subsection II-B). Since $\xi$ satisfies Assumption 1, the above inequality leads to

[TABLE]

We next define

[TABLE]

for $k$ and $K\in{\bf N}_{0}$ such that $K\geq k\geq 0$. Then, it satisfies

[TABLE]

and it follows from (1) that

[TABLE]

On the other hand, (24) also implies that the sequence of

[TABLE]

with respect to $K\ (\geq k)$ for each fixed $k$ is monotonically non-decreasing under the semi-order relation based on positive semidefiniteness (i.e., $E[P^{K}_{k}]\leq E[P^{K+1}_{k}]$). In addition, it follows from (23) (and $a\geq 1$) that

[TABLE]

whose right-hand side converges to a constant matrix as $K\rightarrow\infty$. Hence, this sequence also converges to a constant matrix as $K\rightarrow\infty$. Since this constant matrix does not depend on $k$ because of Assumption 1, we denote it by $P$, which is obviously positive definite. Then, letting $K\rightarrow\infty$ in (26) leads to

[TABLE]

which holds for every $k\in{\bf N}_{0}$. Hence, (6) with $\lambda$ replaced by $\lambda_{\epsilon}\ (<1)$ is satisfied, which means by definition that the system is quadratically stable. This completes the proof. ∎

As stated at the beginning of this section, no equivalence similar to that in the above theorem holds in the case with the usual deterministic linear time-varying systems. Hence, this equivalence cannot be obtained without dealing with randomness behind our system and thus the relevant stability definitions for the system viewed as stochastic systems appropriately. In particular, Assumption 1 played a crucial role in showing such equivalence. To see this, let us temporarily consider a Markov chain $\xi$ (which fails to satisfy Assumption 1) and the associated system (1), which can be seen as the so-called Markov jump linear system [13]. Then, as is well known, the necessary and sufficient condition for exponential stability of the system can be described only with the mode-dependent Lyapunov matrix; this is true even when the Markov chain behind the system is time-homogeneous (i.e., stationary) and ergodic. Hence, the quadratic stability defined with a constant Lyapunov matrix cannot be equivalent to exponential stability in such a case. This in turn implies that assuming $\xi$ is stationary and ergodic is insufficient for showing the equivalence between quadratic stability and exponential stability, and thus, Assumption 1 is indeed essential. In addition, it is also noted that deterministic LTI systems can be seen as a special case of our systems with $\xi$ satisfying Assumptions 1 and 2 if we restrict our attention to the distribution of $\xi_{k}$ that can take only a single value. Since the definitions of stability in this paper immediately reduce to those for deterministic LTI systems in that case, and since their equivalence is known to hold, our results can be seen as a stochastic extension of such conventional results.

IV Stability Analysis Based on Lyapunov Inequality

Theorems 1 and 2 in the preceding section showed the complete equivalence of the three stability notions defined in Section II for system (1) under Assumptions 1 and 2. Since the definition of quadratic stability is, unlike the other two, expected to be compatible with the analysis based on Lyapunov inequalities, we deal with this stability notion and discuss the corresponding Lyapunov inequality in this section.

IV-A Lyapunov Inequality for Quadratic Stability

We first show the following theorem, which gives key inequality conditions for stability analysis.

Theorem 3

Suppose $\xi$ satisfies Assumption 1 and $A(\xi_{k})$ satisfies Assumption 2. The following three conditions are equivalent.

The system (1) is quadratically stable. 2. 2.

There exist $P\in{\bf S}^{n\times n}_{+}$ and $\lambda\in(0,1)$ such that

[TABLE] 3. 3.

There exists $P\in{\bf S}^{n\times n}_{+}$ such that

[TABLE]

Proof:

1$\Rightarrow$2: Taking $k=0$ in inequality (6) implies

[TABLE]

which is nothing but (30).

2$\Rightarrow$1: Since $\xi$ satisfies Assumption 1, (30) implies

[TABLE]

Since $x_{k}$ and $A(\xi_{k})$ are independent of each other, this further implies

[TABLE]

which is nothing but (6).

2$\Leftrightarrow$3: Adding $(1-\lambda^{2})P>0$ to (30) immediately leads to (31). The opposite assertion is obvious. ∎

If $A(\xi_{0})$ is deterministic, then (31) obviously reduces to the usual Lyapunov inequality for deterministic linear systems. Hence, (31) is a natural extension of the usual Lyapunov inequality for the stochastic systems case. The well-definedness of the expectation in the Lyapunov inequality (31) is ensured under Assumption 2.

In addition, the proof on the equivalence between 1) and 2) of the above theorem implies that, for each $\lambda$ and every $P>0$, (30) holds if and only if (6) holds. This implies that the decay rate of the Lyapunov function in the definition of quadratic stability can be evaluated in a necessary and sufficient sense through (30). This, together with the proof of Theorem 2, further implies that we can also evaluate the convergence rate of the sequence $(\sqrt{E[\|x_{k}\|^{2}]})_{k\in{\bf N}_{0}}$ (i.e., minimal $\lambda$ satisfying (5)) through (30). Hence, the alternative representation (30) of the Lyapunov inequality is also useful.

IV-B Connections to Relevant Results

Since our system description covers a wide class of discrete-time linear systems with stochastic dynamics, the associated results can be seen as a generalization of some existing results. For instance, the following cases are relevant to our study.

Case of Systems with State-Multiplicative Noise: Let us consider the $Z$-dimensional stochastic process $\xi$ satisfying Assumption 1 and

[TABLE]

where $v_{i}\in{\bf R}_{+}\ (i=1,\ldots,Z)$ are given constants. For $\xi_{k}=[\xi_{1k},\ldots,\xi_{Zk}]^{T}$, let us further consider the system (1) with

[TABLE]

where $A_{i}\in{\bf R}^{n\times n}\ (i=0,\ldots,Z)$ are given constant matrices. This class of stochastic systems are called systems with state-multiplicative noise; obviously, this class is a special case of our systems. Hence, it readily follows from Theorem 3 that the system is quadratically (i.e., exponentially) stable if and only if there exists $P\in{\bf S}^{n\times n}_{+}$ such that

[TABLE]

This LMI condition is nothing but that in Chapter 9 of [7].

Case of Switched Systems with i.i.d. Switching Signal: Let us next consider the $1$-dimensional stochastic process $\xi$ satisfying Assumption 1 and

[TABLE]

where $D(d,p)$ denotes the discrete distribution such that the event $\xi_{k}=i$ occurs with probability $p_{i}$ for each $i=1,\ldots,S$. Let us further consider the system (1) with

[TABLE]

where $A_{[i]}\in{\bf R}^{n\times n}\ (i=1,\ldots,S)$ are given constant matrices. We see that the value $A_{[\xi_{k}]}$ is switched in accordance with the i.i.d. switching signal $\xi$, and hence, the above system is a switched system with an i.i.d. switching signal. Since this system is also a special case of our systems, we can see that the system is quadratically stable if and only if there exists $P\in{\bf S}^{n\times n}_{+}$ such that

[TABLE]

This LMI condition is nothing but that in Chapter 3 of [13] (see Corollary 3.26).

IV-C LMI Optimization

We next discuss how to solve the Lyapunov inequality (30) or (31) for stability analysis of system (1). As in the preceding subsection, our Lyapunov inequality readily reduces to standard LMIs with given deterministic (scalars and) matrices in cases with some specific systems. In the general case, however, the form of inequalities (30) and (31), in which the decision variable $P$ is contained in the operation of expectation, makes it nontrivial to solve them. This issue can be resolved as follows.

Let us first define

[TABLE]

whose elements cover all the second degree products of the elements of $A(\xi_{0})$. $E[A_{\rm e}(\xi_{0})]$ is well-defined by Lemma 1 under Assumption 2 and becomes a positive semidefinite matrix. Let us further take $\bar{A}\ (\in{\bf R}^{n^{2}\times n^{2}})$ such that

[TABLE]

and introduce the following partitioning of $\bar{A}$.

[TABLE]

Then, for

[TABLE]

the matrix

[TABLE]

with the decision variable $P$ can be confirmed to coincide with $E[A(\xi_{0})^{T}PA(\xi_{0})]$. In addition, another representation of $E[A(\xi_{0})^{T}PA(\xi_{0})]$ can be also given by

[TABLE]

for $\bar{A}_{\rm e2}=E[A_{\rm e2}(\xi_{0})]\in{\bf R}^{n\times n^{3}}$, where

[TABLE]

under the partitioning $A(\xi_{0})=:[a_{1},a_{2},\ldots,a_{n}]$ ($\xi_{0}$ is omitted in the column random vectors for notation simplicity).

Although (45) has a form compatible with the extension toward stabilization synthesis discussed in the next section, (46) has the advantage that we do not need to decompose matrices as in (42). The above arguments can be summarized by the following lemma.

Lemma 2

For given $P$, the expectation $E[A(\xi_{0})^{T}PA(\xi_{0})]$ is equivalent to (45) and (46).

The important point here is that in both (45) and (46), the decision variable $P$ has been taken out from the expectation operation. The implication is that once we calculate $\bar{A}^{\prime}$ in (45) or $\bar{A}_{\rm e2}$ in (46), we can then solve (30) and (31) as the standard linear matrix inequalities (LMIs).

V Stabilization State Feedback Synthesis Based on Lyapunov Inequality

In this section, we discuss stabilization state feedback synthesis based on the Lyapunov inequality condition derived in the preceding section.

V-A Problem of Stabilization State Feedback Synthesis

We first state the synthesis problem to be tackled in this section. Let us consider the $Z$-dimensional process $\xi$ satisfying Assumption 1 and the associated system

[TABLE]

where $x_{k}\in{\bf R}^{n}$, $u_{k}\in{\bf R}^{m}$, $A_{\rm op}:{\boldsymbol{\mit\Xi}}\rightarrow{\bf R}^{n\times n}$ and $B_{{\rm op}}:{\boldsymbol{\mit\Xi}}\rightarrow{\bf R}^{n\times m}$. On the coefficient matrices of the above system, we make the following assumption similar to Assumption 2.

Assumption 3

The squares of elements of $A_{\rm op}(\xi_{k})$ and $B_{\rm op}(\xi_{k})$ are all Lebesgue integrable.

Let us consider the state feedback

[TABLE]

with the static time-invariant gain $F\in{\bf R}^{m\times n}$. The closed-loop system can be described by (1) with

[TABLE]

Note that if $A_{\rm op}(\xi_{k})$ and $B_{\rm op}(\xi_{k})$ satisfy Assumption 3 then the above $A(\xi_{k})$ also satisfies Assumption 2 (for each fixed $F$) by Lemma 1. This section studies the synthesis problem of $F$ such that the closed-loop system is quadratically stable.

V-B LMI for Synthesis

For a given $F\in{\bf R}^{m\times n}$, it readily follows from Theorem 3 that the closed-loop system is quadratically stable if and only if there exists $P\in{\bf S}^{n\times n}_{+}$ such that

[TABLE]

Hence, our synthesis problem reduces to that of searching for $F$ such that there exists $P>0$ satisfying the above inequality. Since the inequality not only involves the expectation operation but also is nonlinear in the decision variables $P$ and $F$, it is more difficult to deal with than (31) about the analysis. Fortunately, however, a technique similar to (45) can indeed lead us to an alternative representation of (51) that is compatible with the Schur complement technique [7].

To see this, let us first define

[TABLE]

whose elements cover all the second order products of the elements of $[A_{\rm op}(\xi_{0}),B_{\rm op}(\xi_{0})]$. $E[G_{\rm e}(\xi_{0})]$ is well-defined under Assumption 3 and becomes a positive semidefinite matrix. Let us further take $\bar{G}\ (\in{\bf R}^{(n+m)n\times(n+m)n})$ such that

[TABLE]

and introduce the following partitioning of $\bar{G}$.

[TABLE]

Then, for

[TABLE]

the matrix

[TABLE]

with the decision variables $P$ and $F$ can be confirmed to coincide with $E[(A_{\rm op}(\xi_{0})+B_{\rm op}(\xi_{0})F)^{T}P(A_{\rm op}(\xi_{0})+B_{\rm op}(\xi_{0})F)]$. Hence, once we calculate $\bar{G}^{\prime}_{A}$ and $\bar{G}^{\prime}_{B}$, the inequality condition (51) can be dealt with as a standard matrix inequality; in particular, the resulting inequality has a form compatible with the Schur complement technique.

Since $P\otimes I_{(n+m)n}>0$ for $P\in{\bf S}^{n\times n}_{+}$, the above arguments lead us to the following lemma.

Lemma 3

For given $P\in{\bf S}^{n\times n}_{+}$ and $F\in{\bf R}^{m\times n}$, (51) holds if and only if

[TABLE]

where $\ast$ denotes the transpose of the lower left block in the matrix.

This lemma, together with the congruence transformation with ${\rm diag}(X,X\otimes I_{(n+m)n})$ for $X=P^{-1}$ and the change of variables $Y=FX$, further leads us to the following theorem about the synthesis.

Theorem 4

Suppose $\xi$ satisfies Assumption 1 and $A_{\rm op}(\xi_{k})$ and $B_{\rm op}(\xi_{k})$ satisfy Assumption 3. There exists a gain $F$ such that the closed-loop system (1) with (50) is quadratically stable if and only if there exist $X\in{\bf S}^{n\times n}_{+}$ and $Y\in{\bf R}^{m\times n}$ satisfying

[TABLE]

for $\bar{G}^{\prime}_{A}$ and $\bar{G}^{\prime}_{B}$ defined by (52)–(56). In particular, $F=YX^{-1}$ is one such stabilization gain.

Although the above theorem is derived from (31) without $\lambda$, the same technique can be applied also to (30) with $\lambda$, which leads to the following corollary.

Corollary 2

Suppose $\xi$ satisfies Assumption 1 and $A_{\rm op}(\xi_{k})$ and $B_{\rm op}(\xi_{k})$ satisfy Assumption 3. There exists a gain $F$ such that the corresponding closed-loop system is quadratically stable if and only if there exist $X\in{\bf S}^{n\times n}_{+}$, $Y\in{\bf R}^{m\times n}$ and $\lambda\in(0,1)$ satisfying

[TABLE]

In particular, $F=YX^{-1}$ is one such stabilization gain.

If we aim not only at stabilizing the closed-loop system but also at minimizing $\lambda$ in (6) (which corresponds to the convergence rate related to the definition of exponential stability by Theorem 2), this corollary will play an important role.

VI Numerical Examples

This section is devoted to numerical examples. We first numerically demonstrate with a simple example that the Lyapunov inequality (30) gives a necessary and sufficient condition for quadratic stability (and thus exponential stability) of system (1) as indicated by Theorems 2 and 3. Then, we provide a more challenging example for motivating our study, in which the stabilization state feedback is designed for the discrete-time system obtained through discretizing a continuous-time deterministic linear system with a randomly time-varying sampling interval.

VI-A *Demonstration of Strictness in Stability Analysis Based on

Lyapunov Inequality*

Let us consider the $2$-dimensional stochastic process $\xi$ that satisfies Assumption 1 and is given by the sequence of $\xi_{k}=[\xi_{1k},\xi_{2k}]^{T},\ \xi_{1k}\sim N(\mu,\sigma^{2})\ (\mu=0,\sigma=0.2),\ \xi_{2k}\sim U(\underline{d},\overline{d})\ (\underline{d}=-0.5,\overline{d}=0.5)$, where $N(\mu,\sigma^{2})$ and $U(\underline{d},\overline{d})$ respectively denote the normal distribution with mean $\mu$ and standard deviation $\sigma$ and the continuous uniform distribution with minimum $\underline{d}$ and maximum $\overline{d}$. Let us further consider the stochastic system (1) with

[TABLE]

Through numerical stability analysis of this system, we discuss the strictness of our Lyapunov inequality condition.

We first search for the minimal $\lambda$ such that there exists $P>0$ satisfying (30). As stated in Lemma 2, the matrix (45) is an alternative representation of the expectation $E[A(\xi_{0})^{T}PA(\xi_{0})]$ in (30). Hence, once we calculate $\bar{A}^{\prime}$ in (45) for the above $A(\xi_{0})$, it readily follows that we can solve (30) as an LMI for each fixed $\lambda$. This enables us to achieve the aforementioned minimization through a bisection method with respect to $\lambda$; the resulting minimum is expected to correspond to the convergence rate with respect to $\left(\sqrt{E[||x_{k}||^{2}]}\right)_{k\in{\bf N}_{0}}$ by Theorems 2 and 3. We performed calculation of $\bar{A}^{\prime}$ with MATLAB and Symbolic Math Toolbox, and minimized $\lambda$ with MATLAB, YALMIP [21] and SDPT3 [22]. Then, the minimal $\lambda$ became 0.9219$\ (<1)$, which implies exponential stability of the system by Theorems 2 and 3.

We next confirm that the above minimal $\lambda$ indeed corresponds to the convergence rate with respect to $\left(\sqrt{E[||x_{k}||^{2}]}\right)_{k\in{\bf N}_{0}}$, through calculating its estimate $\lambda_{\rm est}$ with $N_{\rm s}$ sample paths of $\xi$. For $N_{\rm s}=10^{5}$, such sample-based estimation of $\sqrt{E[||x_{k}||^{2}]}$ provided us with the time response shown in Fig. 1, where we took $x_{0}=[1,0,0]^{T}$ as the initial state of the system. As we can see from the figure, the estimate of $\sqrt{E[||x_{k}||^{2}]}$ decays with an almost constant rate after the elapse of sufficient time. The decay rate $\lambda_{\rm est}$ obtained from the data at $k=50$ and $100$ in Fig. 1 was 0.9213 (similar results were obtained regardless of the initial state under the same sample of $\xi$). Since this value is close to the above minimization result, it numerically suggests that the minimal $\lambda$ obtained with the Lyapunov inequality corresponds to the true convergence rate.

VI-B *Stabilization of Discrete-Time System Obtained under

Randomly Time-Varying Sampling Interval*

Let us next consider the sampled-data system, shown in Fig. 2, consisting of the continuous-time deterministic linear unstable system $P_{\rm c}$ given by

[TABLE]

the static time-invariant state feedback gain $F$ to be designed, and the sampler ${\cal S}$ and the zero-order hold ${\cal H}$ running with the sampling instants $t_{k}\ (k\in{\bf N}_{0})$, where

[TABLE]

The relation between the continuous-time signals and the discrete-time signals in Fig. 2 is described as follows.

[TABLE]

For such a sampled-data system, we assume that the sampling interval $h_{k}=t_{k+1}-t_{k}$ is randomly time-varying (i.e., the random case of aperiodic sampling [17, 18]) and given by $h_{k}=h(\xi_{k})=0.01+\xi_{k}$ with the $1$-dimensional stochastic process $\xi$ that satisfies Assumption 1 and $\xi_{k}\sim{\rm Exp}(\nu)\ (\nu=20)$, where ${\rm Exp}(\nu)$ denotes the exponential distribution with expectation $1/\nu$. In this subsection, we consider designing $F$ stabilizing this sampled-data stochastic system; if we focus only on the signal values at the sampling instants, this synthesis problem reduces to that of designing a state feedback (49) stabilizing the discrete-time stochastic system (48) with the random coefficients

[TABLE]

For the above discrete-time system, we searched for a solution of (60) minimizing $\lambda$ with MATLAB, Symbolic Math Toolbox, YALMIP and SDPT3, where matrix exponentials were dealt with through the second-order Padé approximation in the computation. Then, we obtained the gain $F=\left[2.9242,4.9123,-10.0501\right]$ with $\lambda=0.9193$, which implies the stability of the corresponding discrete-time closed-loop system by Corollary 2. Since our control approach is developed for discrete-time stochastic systems, it can only ensure the convergence of the state of the sampled-data system (in the stochastic sense) with respect to the sampling instants immediately. Fortunately, however, the responses of the continuous-time signals $x_{\rm c}$ and $u_{\rm c}$ in the present sampled-data system (with the above $F$) indeed converged to zero in the simulations of the authors; Fig. 3 shows the overlays of the responses of $x_{\rm c}$ and $u_{\rm c}$ generated with 100 sample paths of $\xi$ and the initial state $x_{\rm c}(t_{0})=[1,0,0]^{T}$. Since there is virtually no limitation on the class of continuous-time linear systems (and that of the distributions of $h_{k}$) in the above synthesis, other synthesis problems could also be solved in a similar fashion. This suggests strong potential of the proposed approach.

VII Conclusion

In this paper, we first showed that asymptotic stability, exponential stability and quadratic stability are all equivalent for discrete-time linear systems with stochastic dynamics under the assumption that the underlying process $\xi$ has an i.i.d. property. Then, we discussed a Lyapunov inequality that can characterize stability in a necessary and sufficient sense. Our Lyapunov inequality readily reduces to standard LMIs for some subclasses of stochastic systems. In the general case, however, the original form of the inequality seemed unsuitable for numerical stability analysis because it must be solved for decision variables contained in the expectation operation. Hence, we also provided an idea to solve the inequality as a standard LMI even in the general case; this idea was also used in the extension of the Lyapunov inequality condition toward stabilization state feedback synthesis.