Interplay Between Transmission Delay, Average Data Rate, and Performance in Output Feedback Control over Digital Communication Channels

TL;DR

This paper analyzes how transmission delay affects the minimal data rate needed for control performance over digital channels, using information theory and demonstrating near-optimal practical coding schemes.

Contribution

It introduces an information-theoretic framework to quantify the minimal data rate considering delay, and shows how delay increases the data rate requirement for control.

Findings

Delay increases the data rate needed for desired performance.

Lower bounds are derived using directed information and LTI filters.

Practical schemes achieve rates close to theoretical bounds.

Abstract

The performance of a noisy linear time-invariant (LTI) plant, controlled over a noiseless digital channel with transmission delay, is investigated in this paper. The rate-limited channel connects the single measurement output of the plant to its single control input through a causal, but otherwise arbitrary, coder-controller pair. An infomation-theoretic approach is utilized to analyze the minimal average data rate required to attain the quadratic performance when the channel imposes a known constant delay on the transmitted data. This infimum average data rate is shown to be lower bounded by minimizing the directed information rate across a set of LTI filters and an additive white Gaussian noise (AWGN) channel. It is demonstrated that the presence of time delay in the channel increases the data rate needed to achieve a certain level of performance. The applicability of the results is verified through a numerical example. In particular, we show by simulations that when the optimal filters are used but the AWGN channel (used in the lower bound) is replaced by a simple scalar uniform quantizer, the resulting operational data rates are at most around 0.3 bits above the lower bounds.

Figures

Equations

$$\begin{array}{rl}& \underset{n\to \infty }{\lim }\frac{1}{n}\sum _{i=1}^n\sum _{j=1}^n{|{[P_n-Q_n]}_{i,j}|}^2=0\\ & |{\lambda }_{max}(P_n)|,|{\lambda }_{max}(Q_n)|\le \varrho ,\forall n\in \mathbb{N}\end{array}$$

$$\left[\begin{array}[]{c}z\\ y\end{array}\right]=\left[\begin{array}[]{lr}G_{11}&G_{12}\\ G_{21}&G_{22}\end{array}\right]\left[\begin{array}[]{c}w\\ u\end{array}\right],$$

$$\mathcal{R}\triangleq \underset{k\to \infty }{\lim }\frac{1}{k}\sum _{i=0}^{k-1}R(i),$$

$$y_q(k)=E_k(y^k,{\eta }_e^k),$$

$$u(k)=\{\begin{array}{ll}{D}_k({\eta }_d^k),& 0\le k<h,\\ D_k(y_q^{k-h},{\eta }_d^k),& k\ge h.\end{array}$$

$$\mathcal{R}(D)=\underset{{\sigma }_z^2\le D}{\inf }\mathcal{R}$$

$$\mathcal{R}\ge I_{\infty }^{(h)}(y\to u)=\underset{k\to \infty }{\lim }\frac{1}{k}{\Sigma }_{i=0}^{k-1}I(u(i);y^{i-h}\mid u^{i-1}),$$

$$\begin{array}{rl}& {\Sigma }_{i=0}^{k-1}I(u(i);y^{i-h}\mid u^{i-1})\stackrel{\text{makebox}[0.0pt]\text{mbox}(a)}{=}I(x(0),w^{k-1};u^{k-1})\\ & \stackrel{\text{makebox}[0.0pt]\text{mbox}(b)}{\ge }I(x(0),w^{k-1};u_G^{k-1})\\ & \stackrel{\text{makebox}[0.0pt]\text{mbox}(c)}{=}{\Sigma }_{i=0}^{k-1}I(x(0),w^{k-1};u_G(i)\mid u_G^{i-1})\\ & \stackrel{\text{makebox}[0.0pt]\text{mbox}(d)}{=}{\Sigma }_{i=0}^{k-1}I(x(0),w^i;u_G(i)\mid u_G^{i-1})\\ & \stackrel{\text{makebox}[0.0pt]\text{mbox}(e)}{=}{\Sigma }_{i=0}^{k-1}[I(x(0),w^i,y_G^{i-h};u_G(i)\mid u_G^{i-1})\\ & -I(y_G^{i-h};u_G(i)\mid u_G^{i-1},x(0),w^i)]\\ & \stackrel{\text{makebox}[0.0pt]\text{mbox}(f)}{\ge }{\Sigma }_{i=0}^{k-1}[I(x(0),w^{i-1},y_G^{i-h};u_G(i)\mid u_G^{i-1})\\ & -I(y_G^{i-h};u_G(i)\mid u_G^{i-1},x(0),w^i)]\\ & \stackrel{\text{makebox}[0.0pt]\text{mbox}(g)}{=}{\Sigma }_{i=0}^{k-1}I(x(0),w^{i-1},y_G^{i-h};u_G(i)\mid u_G^{i-1})\\ & \stackrel{\text{makebox}[0.0pt]\text{mbox}(h)}{=}{\Sigma }_{i=0}^{k-1}I(y_G^{i-h};u_G(i)\mid u_G^{i-1})\\ & +I(x(0),w^{i-1};u_G(i)\mid u_G^{i-1},y_G^{i-h})\\ & \stackrel{\text{makebox}[0.0pt]\text{mbox}(i)}{=}{\Sigma }_{i=0}^{k-1}I(y_G^{i-h};u_G(i)\mid u_G^{i-1}),\end{array}$$

$${{I_{\infty}^{(h)}}(y\to{u})}=\frac{1}{4\pi}\int_{-\pi}^{\pi}\log\big{(}\frac{{S_{\check{u}}}(e^{j\omega})}{{\sigma}_{\psi}^{2}}\big{)}d\omega,$$

$$\psi (k)\triangleq u(k)-\tilde{u}(k),\tilde{u}(k)\triangleq E[u(k)\mid y^{k-h},u^{k-1}].$$

$$\begin{array}{rl}& I(u(i);y^{i-h}\mid u^{i-1})\stackrel{\text{makebox}[0.0pt]\text{mbox}(j)}{=}h(u(i)\mid u^{i-1})-h(u(i)\mid y^{i-h},u^{i-1})\\ & \stackrel{\text{makebox}[0.0pt]\text{mbox}(k)}{=}h(u(i)\mid u^{i-1})-h(\psi (i)+\tilde{u}(i)\mid y^{i-h},u^{i-1})\\ & \stackrel{\text{makebox}[0.0pt]\text{mbox}(l)}{=}h(u(i)\mid u^{i-1})-h(\psi (i)\mid y^{i-h},u^{i-1})\\ & \stackrel{\text{makebox}[0.0pt]\text{mbox}(m)}{=}h(u(i)\mid u^{i-1})-h(\psi (i)),\end{array}$$

$$\begin{array}{rl}& I_{\infty }^{(h)}(y\to u)=\underset{k\to \infty }{\lim }\frac{1}{k}\sum _{i=0}^{k-1}\{h(u(i)\mid u^{i-1})-h(\psi (i))\}\\ & \stackrel{\text{makebox}[0.0pt]\text{mbox}(n)}{=}\underset{k\to \infty }{\lim }\frac{1}{k}h(u^{k-1})-\underset{k\to \infty }{\lim }h(\psi (k))\\ & \stackrel{\text{makebox}[0.0pt]\text{mbox}(o)}{=}\frac{1}{4\pi }{\int }_{-\pi }^{\pi }\log (2\pi e{S}_{\check{u}}(e^{j\omega }))d\omega -\frac{1}{2}\log (2\pi e{\sigma }_{\psi }^2),\end{array}$$

$$u^{\prime }=J{z}^{-h}r,r=t+\eta ,t=B\mathrm{diag}\{z^{-1},1\}\left[\begin{array}{c}r\\ y^{\prime }\end{array}\right],$$

$${{\mathcal{R}}(D)}\geq{{{\vartheta}^{\prime}}_{u}}(D)\triangleq\inf_{{{\sigma}_{z^{\prime}}^{2}}\leq{D}}{\frac{1}{4\pi}\int_{-\pi}^{\pi}\log\big{(}\frac{{S_{u^{\prime}}}(e^{j\omega})}{{\sigma}_{\eta}^{2}}\big{)}},$$

$$\displaystyle\begin{split}\mathcal{R}&\geq{{I_{\infty}^{(h)}}(\hat{y}\to\hat{u})}\geq{I_{\infty}^{(h)}}({\hat{y}_{G}}\to\hat{u}_{G})\\ &=\frac{1}{4\pi}\int_{-\pi}^{\pi}\log\big{(}\frac{{S_{\breve{u}}}(e^{j\omega})}{{\sigma}_{{\hat{\psi}}_{G}}^{2}}\big{)}d\omega\end{split}$$

$${\hat{u}}_G(k)=L_k({\hat{y}}_G^{k-h},{\hat{u}}_G^{k-1})+{\hat{\psi }}_G(k-h),k\in {\mathbb{N}}_0$$

$${\hat{u}}_G^k=Q_k{\hat{\psi }}_G^{k-h}+P_k{\hat{y}}_G^{k-h},k\in {\mathbb{N}}_0.$$

$$\displaystyle\begin{split}{{I_{\infty}^{(h)}}(y^{\prime}\to{u^{\prime}})}&=\frac{1}{4\pi}\int_{-\pi}^{\pi}\log\big{(}\frac{{S_{u^{\prime}}}(e^{j\omega})}{{\sigma}_{\eta}^{2}}\big{)}d\omega\\ &=\frac{1}{4\pi}\int_{-\pi}^{\pi}\log\big{(}\frac{{S_{\breve{u}}}(e^{j\omega})}{{\sigma}_{{\hat{\psi}}_{G}}^{2}}\big{)}d\omega,\end{split}$$

$${\vartheta}_{r}^{{}^{\prime}}(B,J,{{\sigma}_{{\eta}}^{2}})\triangleq\frac{1}{4\pi}\int_{-\pi}^{\pi}\log\big{(}\frac{{S_{r}}(e^{j\omega})}{{\sigma}_{\eta}^{2}}\big{)},$$

$$\begin{array}{rl}& {\vartheta }_r^{{}^{\prime }}(B_1,J_1,{\sigma }_{\eta }^2)={\vartheta }_r^{{}^{\prime }}(B_2,J_2,{\sigma }_{\eta }^2)\\ & =\frac{1}{2}\log (1+\frac{{\sigma }_t^2}{{\sigma }_{\eta }^2}){\mid }_{(B,J)=(B_2,J_2)}-\rho \end{array}$$

$$M={(1-B_r{z}^{-1}-G_{22}J{z}^{-h}{B}_y)}^{-1}.$$

$$\begin{array}{r}{J}_2=z^{d_1}{J}_1{V}^{-1},B_{y2}=z^{-d_1}{B}_{y1}\\ B_{r2}=z(1-(1-B_{r1}{z}^{-1})V^{-1}),\end{array}$$

$$\begin{array}{rl}{T}_2=\mathrm{diag}\{z^{d_1}I,z^{d_1}I,& V,z^{d_1}I\}\times T_1\times \\ & \mathrm{diag}\{I,z^{-d_1}I,z^{-d_1}I,z^{-d_1}I\}.\end{array}$$

$$\begin{array}{rl}{\hat{\Gamma }}_{r_1}& \triangleq {\Gamma }_{r_1}\prod _{i=1}^{m}z{(z-{\delta }_i)}^{-1}\\ V_{\zeta }& \triangleq {({\hat{\Gamma }}_{r_1})}^{-1}{\hat{\Gamma }}_{r_1}(\infty )\prod _{i=1}^{m}z{(z-\zeta {\delta }_i)}^{-1}.\end{array}$$

$$T=\left[\begin{array}{cccc}{G}_{12}J{z}^{-h}M& G_{11}+G_{12}J{z}^{-h}{B}_{y}M{G}_{21}& G_{12}{z}^{-h}(1-B_r{z}^{-1})M& G_{12}J{z}^{-h}{B}_{y}M\\ G_{22}J{z}^{-h}M& G_{21}(1-B_r{z}^{-1})M& G_{22}{z}^{-h}(1-B_r{z}^{-1})M& G_{22}J{z}^{-h}{B}_{y}M\\ M& G_{21}{B}_{y}M& G_{22}{z}^{-h}{B}_{y}M& B_{y}M\\ JM& G_{21}J{B}_{y}M& (1-B_r{z}^{-1})M& J{B}_{y}M\end{array}\right]$$

$$\mathcal{R}(D)\ge \frac{1}{2}\log (1+{\phi }^{\prime }(D)),{\phi }^{\prime }(D)\triangleq \underset{{\sigma }_{z^{\prime }}^2\le D}{\inf }\frac{{\sigma }_t^2}{{\sigma }_{\eta }^2},$$

$${{\vartheta }^{\prime }}_u(D)+\zeta \ge {\vartheta }_r^{{}^{\prime }}(B_{\zeta },1,{\sigma }_{{\eta }_{\zeta }}^2).$$

$${{\vartheta }^{\prime }}_u(D)+\zeta \ge \frac{1}{2}\log (1+\frac{{\sigma }_t^2}{{\sigma }_{\eta }^2}){\mid }_{(B,J,{\sigma }_{\eta }^2)=({\tilde{B}}_{\zeta },{\tilde{J}}_{\zeta },{\sigma }_{{\eta }_{\zeta }}^2)}-\rho$$

$${G_{a}}=\left[\begin{array}[]{lr}G_{11}&{z^{-h}}{G_{12}}\\ {G_{21}}&{z^{-h}}{G_{22}}\end{array}\right],$$

$$\begin{array}{rl}\frac{{\sigma }_t^2}{{\sigma }_{\eta }^2}& ={\Vert M-1\Vert }_2^2+{\Vert B_{y}M{G}_{21}\Vert }_2^2{\sigma }_{\eta }^{-2},\\ {\sigma }_{z^{\prime }}^2& ={\Vert G_{11}+G_{12}N{(1-G_{22}N)}^{-1}{G}_{21}\Vert }_2^2+{\Vert G_{12}JM\Vert }_2^2{\sigma }_{\eta }^2,\end{array}$$

Full text

I INTRODUCTION

Taking communication imperfections into account for analysis and design has proved to be an overwhelming topic within the area of control theory during recent years. Among those imperfections, time delay, packet dropout and bit rate constraint (quantization) are prominent ones, which may worsen the performance and even bring destabilization to networked control systems (NCSs) [1],[2].

Rate-constrained NCSs are generally studied form two points of view; control theory and information theory. Regarding the first viewpoint, classical nonlinear control methods are deployed (see, e.g., [3, 4] as early results). As a recent contribution, [5] obtains stability conditions in terms of the quantizer’s step size by using sliding mode analysis. For the second point of view, the key idea is extending information-theoretic notions to the case of closed-loop control. Regarding stabilization, early results can be found in [6]. Recently, in [7], mean square stability (MSS) conditions for an unstable human-in-the-loop system are stated in terms of a lower bound on the information rate between control input and system output.

For performance, efforts fall into either control-based or information theory-based approach as well. Extending Bode integral to discrete linear time-periodic multirate systems is carried out in [8]. A packetized predictive control strategy is studied in [9] where the Markov jump linear systems (MJLSs) theory is used for deriving an upper bound on the bit rate required to attain a desired performance level, under the circumstances of entropy-coded dithered quantization (ECDQ111With some abuse of notation, and depending on the context, ECDQ is also to be understood as entropy-coded dithered quantizer) with multiple descriptions.

Studies analyzing system performance from an information-theoretic viewpoint are less abundant in the literature. Primary results are presented in [10]. In this work, for a discrete-time linear time-invariant (LTI) plant, the well-known Bode’s integral is extended to the case with causal rate-limited arbitrary feecback. Along the lines of [10], research reported in [11, 12] has investigated bounds on the minimum data rate which is needed to attain a quadratic performance level in NCSs with delay-free channel. While the design approach proposed in [11] is a second-stage one, coding and control are designed jointly in [12]. For the lower bound, [12] shows that the rate-constrained optimization to find desired infimal data rate over causal but otherwise arbitrary coder-controller pairs, is reduced to a convex SNR-constrained optimization over an auxiliary LTI feedback loop with additive white Gaussian noise (AWGN). Based on the SNR-constrained approach, [12] proposes an ECDQ-based linear coding scheme which leads to an upper bound at most 1.25 bits away from the obtained lower bound. Then the main idea followed in [11, 12] (that is, minimizing directed information rate to get lower bound and entropy coding for upper bound) is applied to LQG control of a fully-observable multiple-input multiple-otput (MIMO) plant in [13]. The authors of [13] deploy semidefinite programming (SDP) to solve corresponding optimization problems.

In this paper, we address output feedback control of an NCS comprised of a noisy LTI plant and a causal encoder-controller-decoder set connected through a noiseless digital channel with a constant transmission delay. More specifically, the problem is obtaining the bounds on minimal average data rate required to guarantee that the steady-state variance of an error signal does not become larger than a certain value. Motivated by its merits such as simplicity and practical appeal, we use the approach pursued in [11, 12] to gain outer bounds and build upon [12]. However, the main departure of this work from [12] is considering a channel which is not delay-free. So, as the first contribution, we rederive fundamental information inequalities of the system under the delay assumption. Secondly, we characterize the trade-off among performance, delay, and minimal desired average data rate. It is shown through a numerical example that greater transmission delay necessitates greater minimal average data rate needed to guarantee achieveing the considered quadratic level of performance. Simulation indicates that by deploying a simple scalar uniform quantizer in the LTI architecture that gives the lower bound, the quadratic performance is attained by operational average data rates at most $0.3$ bits away from the lower bound.

The outline of this work is as follows. Section II presents the notation and some preliminaries. Then the problem of interest is formalized in Section III. Section IV is dedicated to the lower bound characterization. An illustrative numerical simulation is provided in section V. Finally, Section VI concludes the paper.

II NOTATION AND PRELIMINARIES

The set of real numbers is denoted by $\mathbb{R}$ with subset ${\mathbb{R}}^{+}$ as the set of strictly positive real numbers. $\mathbb{N}$ represents the set of natural numbers, based upon which ${\mathbb{N}_{0}}=\mathbb{N}\cup\{0\}$ is defined. Furthermore, $k$ is the time index and for random processes considered in this paper, $k\in{\mathbb{N}_{0}}$ holds. Magnitude and $H_{2}$-norm of a signal are symbolized by $|.|$ and ${\left\lVert.\right\rVert}_{2}$, respectively. Furthermore, the set ${{\mathcal{U}}_{\infty}}$ is defined as the set of all proper and real rational stable transfer functions with inverses that are stable and proper as well. $\mathcal{E}$ denotes the expectation operator and $\log$ stands for the natural logarithm. The entry of matrix $S$ on the $i$-th row and $j$-th column is denoted by $[S]_{i,j}$. Moreover, ${\lambda}_{min}(S)$ and ${\lambda}_{max}(S)$ represent eigenvalues of $S$ with the smallest and largest magnitude, respectively.

All random variables and processes in this paper are assumed to be vector valued, unless otherwise stated. The mutual information between random variables $V$ and $W$ is defined as $I(V;W)\triangleq{H(V)-H(V\mid{W})}$ whit $H(.)$ denoting the entropy. If those variables are conditioned upon another rendom variable, say $Y$, then $I(V;W\mid{Y})\triangleq{H(V\mid{Y})-H(V\mid{W,Y})}$ defines the conditional mutual information between $V$ and $W$ given $Y$ [14]. A random process $\xi$ is said to be asymptotically wide-sense stationary (AWSS) if it satisfies $\lim_{k\to\infty}{\mathcal{E}[\xi(k)]}={\nu}_{\xi}$ and $\lim_{k\to\infty}{\mathcal{E}[(\xi(k+\tau)-\mathcal{E}[\xi(k+\tau)]){(\xi(k)-\mathcal{E}[\xi(k)])}^{T}]}={R}_{\xi}(\tau)$ hold, where ${\nu}_{\xi}$ is a constant. ${C_{\xi}}={R}_{\xi}(0)$ denotes the corresponding steady-state covariance matrix upon which the steady-state variance of $\xi$ is defined as $\sigma_{\xi}^{2}\triangleq\mathrm{trace}({C_{\xi}})$. The covariance matrix for a scalar random sequence ${x}_{1}^{k}\!\!\!\triangleq\!\!{[x(1)\dots x(k)]^{T}}$ is defined as $C_{{x}_{1}^{k}}={\mathcal{E}[({{x}_{1}^{k}}-\mathcal{E}[{{x}_{1}^{k}}]){({{x}_{1}^{k}}-\mathcal{E}[{{x}_{1}^{k}}])}^{T}]}$.

Considering ${P_{n}},{Q_{n}}\in{{\mathbb{R}}^{n\times n}}$ as two square matrices, the sequences $\{P_{n}\}_{n=1}^{\infty}$ and $\{Q_{n}\}_{n=1}^{\infty}$ are asymptotically equivalent if and only if the following holds for finite $\varrho$:

[TABLE]

III PROBLEM STATEMENT AND SETUP

The structure considered in this work can be found in Fig. 1 where $G$ is an LTI plant with $u\in\mathbb{R}$ as control input and $y\in\mathbb{R}$ as sensor output. Moreover, there is a disturbance represented by $w\in\mathbb{R}^{n_{w}}$ and the output signal $z\in\mathbb{R}^{n_{z}}$ upon which the desired performance is characterized. The plant has the following transfer-function matrix description:

[TABLE]

in which every $G_{ij}$ is proper and of suitable dimension. The input alphabet of the channel is represented by $\mathcal{A}$ and is defined as a countable set of prefix-free binary words. Due to the delay, the output of the channel $u_{q}(k)$ follows ${u_{q}}(k)={y_{q}}(k-h)$ for $k\geq{h}$ where $y_{q}(k)$ belongs to $\mathcal{A}$. The average data rate across the channel is specified as follows:

[TABLE]

where $R(i)$ denotes the expected length of the $i$-th binary word ${y_{q}}(i)$. The channel input is provided by the encoder $E$ based on the following dynamics:

[TABLE]

in which ${{\eta_{e}}(k)}$ is the side information at time $k$ at the encoder with ${E_{k}}$ representing an arbitrary (possibly nonlinear or time-varying) deterministic mapping. It should be noted that $\beta^{k}$ is a shorthand for $[\beta(0),\cdots,\beta(k)]$. On the decoder side, we have

[TABLE]

${{D}_{k}}$ is assumed to be an arbitrary deterministic mapping, like ${{E}_{k}}$, and ${\eta_{d}(k)}$ signifies the side information available at the decoder at time $k$. It should be emphasized that $E$ and $D$ in Fig. 1 are possibly time-varying or nonlinear causal systems.

Assumption III.1

The plant $G$ is LTI, proper and free of unstable hidden modes. Moreover, the open-loop transfer function from $u$ to $y$ is single-input single-output (SISO) and strictly proper. The disturbance signal, $w$, is a zero-mean white noise with identity covariance matrix ${C_{w}}=I$ and jointly Gaussian with ${x_{0}}=[x(-h),\cdots,x(0)]^{T}$, the initial condition, having finite differential entropy.

Assumption III.2

Each of processes $\eta_{e}$ and $\eta_{d}$ is jointly independent of $({x_{0}},w)$. So regarding the dynamics of the system, $I(u(k);{y^{k-h}}\mid{u^{k-1}})=0$ holds for $0\leq{k}<h$. Moreover, upon knowledge of $u^{i}$ and $\eta_{d}^{i}$, the decoder is invertible. It means that there exists a deterministic mapping $Q_{i}$ such that $u_{q}^{i}=Q_{i}(u^{i},\eta_{d}^{i})$.

Remark III.1

Regarding the aforementioned setup, invertibility is a reasonable assumption for the decoder. It can be proved that for the architecture of Fig. 1, any encoder and non-invertible decoder pair, with mappings $E_{k}$ and $D_{k}$, can be replaced by another pair with the same input-output relationship and lower average data rate where the decoder is invertible. Due to space limitations, we eliminate the proof.

Definition III.1

A scalar AWSS process $x$ converging to a wide sense stationary process $\bar{x}$ is called strongly asymptotically wide-sense stationary (SAWSS) if asymptotic equivalence holds between their covariance matrices, ${\{C}_{x_{1}^{n}}\}_{n=1}^{\infty}$ and ${\{C}_{{\bar{x}}_{1}^{n}}\}_{n=1}^{\infty}$. Accordingly, in an SAWSS NCS, all covariance and cross-covariance matrices of internal signals are asymptotically equivalent to their counterparts in processes to which they converge.

Now, suppose that Assumption III.1 holds. Let $D_{inf}(h)$ denote the infimum steady-state variance of the output $z$ over all settings $u(k)={\mathcal{K}}_{k}(\gamma^{k})$ for $0\leq{k}<h$ and $u(k)={\mathcal{K}}_{k}(y^{k-h})$ for $k\geq{h}$ with $\gamma^{k}$ independent of $x_{0}$ and $w$. Then the problem of our interest is finding

[TABLE]

for any $D\in{(D_{inf}(h),\infty)}$, where the search is to be restricted to encoders with mapping $E_{k}$ and decoders with mapping $D_{k}$ which satisfy Assumption III.2 and make the NCS of Fig. 1 SAWSS. Moreover, ${{\sigma}_{z}^{2}}$ denotes the steady-state variance of $z$. It can be proved that the optimization problem in (5) is feasible if ${D}\in({D_{inf}}(h),\infty)$ (see Appendix -A).

IV MAIN RESULTS

This section shows that in order to obtain a lower bound on ${\mathcal{R}}(D)$, one can minimize the directed information over an auxiliary coding scheme formed of LTI filters and an AWGN channel with feedback and delay. Regarding this optimization problem, inequalities and identities in [12] will be extended to the case with a channel subject to delay in the following.

Theorem IV.1

For the feedback loop depicted in Fig. 1 and satisfying Assumptions III.1 and III.2, the following holds:

[TABLE]

in which $I(.;.\mid{.})$ indicates conditional mutual information. Moreover, as defined in [15], ${{I_{\infty}^{(h)}}(y\to{u})}$ denotes the directed information rate across the forward channel from $y$ to $u$ with delay $h$. For the proof, see Appendix  -B.

Now, a lower bound can be derived on the directed information across the coding scheme of Fig. 1.

Lemma IV.1

For the NCS of Fig. 1, assume that $(x(0),w,u,y)$ form a jointly second-order set of processes and that Assumptions III.1 and III.2 hold. Moreover, take $y_{G}$ and $u_{G}$ into account as the Gaussian counterparts of $y$ and $u$ where $(x(0),w,u_{G},y_{G})$ are jointly Gaussian with the same first-and second-order (cross-) moments as $(x(0),w,u,y)$. Then ${{I_{\infty}^{(h)}}(y\to{u})\geq{I_{\infty}^{(h)}}({y_{G}}\to{u_{G}})}$.

Proof:

The following inequalities and identities will justify the claim:

[TABLE]

where (a) is obtained by a slight modification in Lemma B.4 of [12] based on the result of Theorem 1 in [15]. Lemma B.1 in [12] will give (b) because as mentioned in Assumption III.1, $x_{0}$ and $w^{k}$ are jointly Gaussian. Moreover, (c) is a result of the chain rule applied on the conditional mutual information. (d) is concluded because regarding (52b) in Theorem B.3 of [12], which holds for the considered NCS in this paper, the Markov chain ${{u}_{G}}(i)-{u_{G}^{i-1}},x(0),{w^{i}}-{{w}_{i+1}^{k-1}}$ holds . Furthermore, the chain rule of conditional mutual information gives (e), and (f) is caused by a property of mutual information. (g) is concluded according to Assumption III.1 and ${{y}_{G}^{i}}$ being a deterministic function of ${u_{G}^{i-1}},x(0)$ and ${{w}^{i}}$. The chain rule will yield (h). Finally, (i) is concluded since regarding (52a) in Theorem B.3 of [12] and Assumption III.1, the validity of the Markov chain ${{u}_{G}}(i)-{u_{G}^{i-1}},{{y}_{G}^{i-h}}-x(0),{{w}^{i-1}}$ is verified. The proof is complete. ∎

What follows will relate the directed information from $y_{G}$ to $u_{G}$ to their corresponding power spectral densities:

Lemma IV.2

Consider $y$ and $u$ as jointly Gaussian AWSS processes. Moreover, suppose that $u$ is SAWSS with $|{{{\lambda}_{min}}(C_{{u}_{1}^{n}})}|\geq{\mu}$, $\forall{n}\in{\mathbb{N}}$ where $\mu>0$. Then the following can be obtained:

[TABLE]

in which $\psi$ is a Gaussian AWSS process with independent samples defined as:

[TABLE]

Moreover, ${S_{\check{u}}}$ represents the steady-state power spectral density of $u$.

Proof:

Having Gaussianity and joint AWSS-ness of $(u,y)$ in mind and based on Theorem 2.4 in [16] with a little modification, we can conclude that $\psi$ is Gaussian and AWSS as well. We start by the following equalities:

[TABLE]

where (j) comes from the definition of mutual information and (k) from the definition of $\psi$. Furthermore, (l) can be concluded based on Property 2 in [12] and (9) where $\tilde{u}(i)$ is a deterministic function of ${y^{i-h}}$ and ${u^{i-1}}$. Due to the independence between $\psi(i)$ and $({y^{i-h}},{u^{i-1}})$, and Property 3 in [12], (m) is obtained. So the directed information rate can therefore be rewritten as follows:

[TABLE]

where (n) follows from the chain rule of the differential entropy and $\psi(k)$ being independent of ${\psi}^{k-1}$. Since the process $u$ is SAWSS with $|{{{\lambda}_{min}}(C_{{u}_{1}^{n}})}|\geq{\mu}$, $\forall{n}\in{\mathbb{N}}$ for some $\mu>0$, Lemma B.5 in [12] will approve the validity of the leftmost term in (o). The rightmost term is self-explanatory because $\psi$ is Gaussian and AWSS. ∎

It follows from Theorem IV.1, Lemma IV.1 and Lemma IV.2 that the rate-performance pair yielded by any encoder-decoder scheme which renders the NCS SAWSS is attainable with a lower rate by a coding scheme comprised of LTI filters and an AWGN noise source. Such a scheme is depicted in Fig. 2.

The NCS of Fig. 2 is defined under the same conditions (Assumption III.1) as the main system in Fig. 1 except for one thing; the arbitrary mappings are replaced by proper LTI filters $B$ and $J$. Moreover, the communication channel is a delayed AWGN channel with noiseless one-sample-delayed feedback. The dynamics of this auxiliary coding scheme can be summarized as follows:

[TABLE]

in which $\eta$ is the AWGN with zero mean and variance ${{\sigma}_{\eta}^{2}}$. This noise is assumed to be independent of $(x_{0},w)$. Additionally, we suppose that the initial state of $B$, $J$, and the delay are deterministic.

Theorem IV.2

For the NCS depicted in Fig. 1 and satisfying Assumptions III.1 and III.2, $\mathcal{R}(D)$ is lower-bounded as follows if ${D}\in({D_{inf}}(h),\infty)$:

[TABLE]

where the feasible set for the optimization problem defining ${{{\vartheta}^{\prime}}_{u}}(D)$ is all LTI filters $B$ and the noise $\eta$ with ${{\sigma}_{{\eta}}^{2}}\in{{\mathbb{R}}_{+}}$ rendering the feedback loop of Fig. 2 internally stable and well-posed when $J=1$. In these expressions, ${{\sigma}_{z^{\prime}}^{2}}$ and ${S_{u^{\prime}}}$ denote the steady-state variance of $z^{\prime}$ and the steady-state power spectral density of $u^{\prime}$ in Fig. 2, respectively.

Proof:

Since $D>{{D}_{inf}}(h)$, there is at least one pair, say $\hat{{E}}$ and $\hat{{D}}$, satisfying Assumption III.2, rendering the NCS of Fig. 1 SAWSS and producing $\hat{z}$, $\hat{y}$ and $\hat{u}$ where ${{{\sigma}_{\hat{z}}^{2}}\leq{D}}$ holds and

[TABLE]

can be concluded based on Theorem IV.1, if the conditions in Lemma IV.1 and Lemma IV.2 are met. It should be noted that ${S_{\breve{u}}}$ denotes the steady-state power spectral density of $\hat{u}_{G}$. A coding scheme comprised of linear filters with a unit-gain noisy channel and delay $h$, as follows, can generate $\hat{y}_{G}$ and $\hat{u}_{G}$ which satisfy those conditions and keep ${{\sigma}_{\hat{z}_{G}}^{2}}$ within $({D_{inf}}(h),\infty)$:

[TABLE]

where ${\hat{\psi}_{G}}(k)$ represents a Gaussian noise with zero mean and independent of $({\hat{y}}_{G}^{k},{\hat{u}}_{G}^{k-1})$. Regarding the causality and linearity of ${L_{k}}$, ${\hat{u}_{G}^{k}}$ can be written as follows:

[TABLE]

According to the causality in (16), joint SAWSS-ness of $({\hat{y}_{G}},\hat{u}_{G})$ and transitivity of asymptotic equivalence for products and sum of the matrices noted in [17], the sequences $\{Q_{k}\}$ and $\{P_{k}\}$ are asymptotically equivalent to sequences of lower triangular Toeplitz matrices. Moreover, $L_{k}$ renders the NCS internally stable and well-posed. With all of this in mind, by setting $J=1$ and $B$ as a concatenation of linear filters with the steady-state behaviour of $L_{k}$ in (15) and considering a variance for $\eta$ equal to ${{\sigma}_{{\hat{\psi}}_{G}}^{2}}$, the system of Fig. 2 will be rendered well-posed and internally stable where ${S}_{u^{\prime}}={S}_{\breve{u}}$ and ${{\sigma}_{{\hat{z}}_{G}}^{2}}={{\sigma}_{{z^{\prime}}}^{2}}$ are resulted.

Then according to Lemma IV.2, the following can be concluded:

[TABLE]

which completes the proof. ∎

Lemma IV.3

Consider the LTI loop of Fig. 2 with fixed ${{\sigma}_{{\eta}}^{2}}\in{{\mathbb{R}}^{+}}$. Define ${\vartheta}_{r}^{{}^{\prime}}$ as follows:

[TABLE]

in which ${S_{r}}$ represents the steady-state power spectral density of $r$. Tehn for any $\rho>0$, upon the existence of the pair $(B,J)=({B_{1}},{J_{1}})$ making the system of Fig. 2 internally stable and well-posed, there exist another pair, comprised of the biproper filter ${J_{2}}$ and ${B_{2}}$, which renders the feedback loop of Fig. 2 internally stable and well-posed, preserves the the steady-state power spectral density of $z^{\prime}$ and satisfies the following:

[TABLE]

Proof:

It is well-known that the system of Fig. 2 is well-posed and internally stable if and only if the transfer function $T$ from ${[\eta,w,{\psi}_{1},{\psi}_{2}]^{T}}$ to ${[z^{\prime},y^{\prime},r,u^{\prime}]^{T}}$ in Fig. 3 belongs to ${\mathcal{R}\mathcal{H}}_{\infty}$. $T$ is calculated as (24) (at the top of the next page) where

[TABLE]

By $T_{i}$ and $r_{i}$, we refer to the transfer-function matrix $T$ and signal $r$ when $B$ and $J$ are set such that $(B,J)=({B_{i}},{J_{i}}),i\in\{1,2\}$. Moreover, ${B}_{yi}$ and ${B}_{ri}$ represent elements of $B$ ($B=[B_{r}\quad B_{y}]$) in the situation where $B={B}_{i}$. Now, consider the following set of filters:

[TABLE]

in which $d_{1}$ indicates the relative degree of $J_{1}$ and $V\in{{\mathcal{U}}_{\infty}}$ is chosen in such a way that $V(\infty)=1$. Consequently, $J_{2}$ is biproper and $T_{2}$ can be written as follows:

[TABLE]

So regarding the definition of $d_{1}$ and properties of $V$, ${T_{2}}\in{\mathcal{R}\mathcal{H}}_{\infty}$ if and only if ${T_{1}}\in{\mathcal{R}\mathcal{H}}_{\infty}$. Moreover, based on the same argument, using $({B_{2}},{J_{2}})$ would give the same power spectral density for $z^{\prime}$ as for the case where $({B_{1}},{J_{1}})$ is utilized. We need ${{\Gamma}_{r_{1}}}$ in $S_{r_{1}}=|{{{\Gamma}_{r_{1}}}}|^{2}$ to be stable, biproper and with all zeros in $\{z\in{\mathbb{C}}:|z|\leq{1}\}$. Let ${\delta_{1}},\dots,{\delta_{m}}$ represent the zeros of ${{\Gamma}_{r_{1}}}$ lying on the unit circle. Now for $\zeta\in{(0,1)}$ we define the following:

[TABLE]

Hence, ${{V}_{\zeta}}\in{{\mathcal{U}}_{\infty}}$ and ${V}_{\zeta}(\infty)=1$ can be deduced for every $\zeta\in{(0,1)}$. By following the same procedure as for the proof of Theorem 5.2 in [11], the existence of $\zeta\in{(0,1)}$ will be shown in such a way that for any ${\rho}>0$, setting $V={V}_{\zeta}$ will give a pair $({B_{2}},{J_{2}})$ that satisfies (19). ∎

Corollary IV.1

If Assumptions III.1 and III.2 hold for the NCS of Fig. 1 and ${D}\in({D_{inf}}(h),\infty)$, then

[TABLE]

where the optimization is done over all LTI filter pairs $(B,J)$ and the noise variance ${{\sigma}_{{\eta}}^{2}}\in{{\mathbb{R}}^{+}}$ making the system in Fig. 2 internally stable and well-posed. Moreover, ${{\sigma}_{t}^{2}}$ and ${{\sigma}_{z^{\prime}}^{2}}$ represent the steady-state variances of $t$ and $z^{\prime}$ , respectively.

Proof:

According to the feasibility of finding ${{{\vartheta}^{\prime}}_{u}}(D)$ (shown in Appendix -A), there exist the triplet $({{B}_{\zeta}},1,{{\sigma}_{{\eta}_{\zeta}}^{2}})$, with ${{B}_{\zeta}}$ a proper LTI filter and ${{\sigma}_{{\eta}_{\zeta}}^{2}}\in{\mathbb{R}^{+}}$, that guarantees ${{\sigma}_{z^{\prime}}^{2}}\leq{D}$ for the system of Fig. 2. Furthermore, based upon the definition of ${{{\vartheta}^{\prime}}_{u}}$ and ${{{\vartheta}^{\prime}}_{r}}$ in (13) and (18), the following can be derived for any $\zeta>0$:

[TABLE]

So regarding Lemma IV.3, since there exist a biproper filter ${{\tilde{J}}_{\zeta}}$ and a proper one ${{\tilde{B}}_{\zeta}}$ making the LTI feedback loop of Fig. 2 internally stable and well-posed and keeping ${\sigma}_{z^{\prime}}^{2}$ intact, the following can be concluded:

[TABLE]

Now the proof is completed by noting that (27) holds for any $\zeta,\rho>0$ ∎

Corollary IV.1 shows that for the general structure of Fig. 1, a lower bound on infimum data rate required to achieve ${{{\sigma}_{z}^{2}}\leq{D}}$ is obtained by minimizing the SNR, ${{{\sigma}_{t}^{2}}/{{\sigma}_{\eta}^{2}}}$, over the auxiliary LTI feedback loop of Fig. 2 subject to ${{{\sigma}_{z^{\prime}}^{2}}\leq{D}}$. To characterize ${{\varphi^{\prime}}(D)}$, we will mostly use properties of linear systems and some results on $H_{2}$ optimization with input-delay.

Consider the auxilary structure of Fig. 4, where except for shifting the delay block to the plant model, which leads to

[TABLE]

the same assumptions as Fig. 2 hold. The NCS of Fig. 4 is internally stable and well-posed if and only if the transfer function ${T_{a}}$ from ${[\eta,w,{\psi}_{1},{\psi}_{2}]^{T}}$ to ${[{z_{a}},{y_{a}},{r_{a}},{u_{a}}]^{T}}$ in Fig. 5 is a member of ${\mathcal{R}\mathcal{H}}_{\infty}$. Regarding (24), it can be easily shown that ${T_{a}}=T$. So the feedback loop of Fig. 4 and Fig. 2 are equivalent in the sense of internal stability and well-posed-ness. Moreover, the SNR and output variance of the NCS depicted in Fig. 2 can be stated in terms of $H_{2}$-norms as follows:

[TABLE]

where $N\triangleq{J{B_{y}}{z^{-h}}{(1-{B_{r}}{z^{-1}})}^{-1}}$. Likewise, the following holds for the structure of Fig. 4:

[TABLE]

in which ${N_{a}}=J{B_{y}}{(1-{B_{r}}{z^{-1}})}^{-1}$ and ${M_{a}}=M$. As seen, comparing (29) and (30) signifies the equalities $({{{\sigma}_{t}^{2}}/{{\sigma}_{\eta}^{2}}})=({{{\sigma}_{t_{a}}^{2}}/{{\sigma}_{\eta}^{2}}})$ and ${{\sigma}_{z^{\prime}}^{2}}={{\sigma}_{z_{a}}^{2}}$. So every triplet $(B,J,{{\sigma}_{\eta}^{2}})$ that can infimize the SNR while making the system output satisfy ${{{\sigma}_{z^{\prime}}^{2}}\leq{D}}$ for the NCS of Fig. 4, can do the same for the the LTI system of our interest, in Fig. 2, and vice versa.

In other words, the NCSs in Fig. 2 and Fig. 4 are equivalent regarding the SNR-perfomance optimization problem in (25) as well. This problem is studied for such feedback systems as auxiliary system of Fig. 4 in [12]. So the approach proposed in [12] can be utilized to derive the lower bound on ${{\mathcal{R}}(D)}$ regarding (25). Consequently, it can be concluded that the problem of finding ${{\varphi^{\prime}}(D)}$ is equivalent to an SNR-constrained optimal control problem which was proved to be convex. As another result, ${{\varphi^{\prime}}(D)}$ being a monotonically decreasing fuction of $D$ can be deduced. All in all, the inteplay between the desired performance, the average data rate and the time delay is characterized through (25), (29) and (30).

V SIMULATION EXAMPLE

Consider the following transfer function representation for the plant $G$ in NCS of Fig. 1:

[TABLE]

where $(x_{0},w)$ satisfies Assumption III.1. Using the results of the previous section, we simulate the lower bound on ${\mathcal{R}}(D)$ obtained in (25) regarding five different values of delay ($h=\{0,1,2,3,4\}$) and for each $h$, over a range of $D>{D_{inf}}(h)$. Fig. 6 demonstrates the behaviour of the lower bound with respect to $D$ and $h$. Additionally, it shows the operational rates when using scalar uniform quantizer for $h=0$ and $h=4$. First, as expected, ${{\varphi^{\prime}}(D)}$ in (25) is a monotonically decreasing function of $D$. Secondly and more importantly, ${D_{inf}}(h)$ increases when $h$ grows. So greater delay yields worse best performance. The most significant outcome is associated with the behaviour of ${\mathcal{R}}(D)$ in (5) with respect to delay. It can be observed from the curves in Fig. 6 that for a fixed $D$, ${{\varphi^{\prime}}(D)}$ is increasing in $h$. Therefore, a delay in the channel forces an increase in the infimum data

rate required to achieve a quadratic level of performance. The greater delay, the higher rate to be spent in order to get a certain level of performance. Another observation is the convergence of the obtained infimal data rates to the minimum rate required for stabilizability as $D\to\infty$. As seen in Fig. 6, high rates are required to attain the ideal non-networked performance ${D_{inf}(h)}$. Nevertheless, in our case, using only 3 bits can result in a ${\sigma}_{z}^{2}$ fairly close to ${D_{inf}(h)}$. It should be noted that in order to make our contribution more clear, the plant in (31) has the same dynamics as the one considered in [12]. So the curve related to the delay-free case ($h=0$) is identical to the lower bound curve obtained in [12]. Along the lines of [11, 12], we now simply replace the AWGN $\eta$ in the independent coding scheme depicted in Fig. 2, by a uniform scalar quantizer in order to assess the operational performance caused by a simple coding scheme. It is interesting to note that the obtained operational average data rate in Fig. 6 is at most around $0.3$ bits away from the derived lower bound at all performance .

VI CONCLUSIONS

In this paper, rate-constrained networked control systems comprising noisy LTI plants, causal but otherwise arbitrary coding-control schemes and digital noiseless communication channels with time delay, have been studied. For such NCSs, a certain level of performance is attainable if and only if the average data rate does not fall below a minimal value. A lower bound on this infimum rate has been obtained. Through a numerical example, it has been illustrated that the channel’s time delay increases the infimum average data rate needed to achieve a prsecribed level of performance. Moreover, by using a simple scalar quantizer, operational average data rates fairly close (around $0.3$ bits) to the lower bound have been obtained.

-A Feasibility proofs

-A1 Feasibility of ${D_{inf}(h)}$

Suppose that in the standard architecture depicted in Fig. 7, $G$, $x_{0}$ and $w$ satisfy Assumption III.1 and $K$ follows $u(k)={{\mathcal{K}}_{k}}({y}^{k-h})$.

Regarding the Gaussianity of $x_{0}$ and $w$ and the fact that $G$ is LTI, it can be implied from some results in [18] that:

[TABLE]

in which ${\sigma}_{z}^{2}$ denotes the variance of output $z$ and ${\kappa}$ is the set of all proper LTI filters which render the system of Fig. 7 internally stable and well-posed. The assumptions considered for $G$ guarantee that finding ${D}_{inf}(h)$ is feasible.

-A2 Feasibility of ${{\vartheta}_{u}^{{}^{\prime}}(D)}$

Since ${D}_{inf}(h)$ can be obtained, for every $\zeta\in(0,{D-{D_{inf}(h)})}$, there exists ${K_{1}}\in{\kappa}$ which gives ${\sigma}_{z_{1}}^{2}\triangleq{{\sigma}_{z}^{2}}\mid_{K=K_{1}}\leq{{D_{inf}(h)}+\zeta<D}$ for the system of Fig. 7. Applying ${K_{1}}$ to this system results in a stable setting which is a special case of the NCS depicted in Fig. 2 with $J=1$ and $r=t={K_{1}}y^{\prime}$ where the steady-state variance of $t$, ${{\sigma}_{t}^{2}}={{\sigma}_{t_{1}}^{2}}$, is finite. Therefore, since ${{K_{1}}\in{\kappa}}$, it can bring internal stability and well-posed-ness to the feedback loop of Fig. 2 in the presence of any additive noise $\eta$ with steady-sate variance ${{\sigma}_{{\eta}}^{2}}\in{{\mathbb{R}}^{+}}$. So ${\sigma}_{z^{\prime}}^{2}={\sigma}_{z_{1}}^{2}+{{\chi}_{z}}{{\sigma}_{\eta}^{2}}$ and ${\sigma}_{t}^{2}={{\sigma}_{t_{1}}^{2}}+{{\chi}_{t}}{{\sigma}_{\eta}^{2}}$ can be concluded, when taking $\eta$ into account as an AWGN with finite variance ${{\sigma}_{{\eta}}^{2}}$ for the system of Fig. 2. It should be noted that ${{\chi}_{t}},{{\chi}_{z}}\geq{0}$ depend only on $K_{1}$. Now by choosing $\zeta=({D-{D_{inf}}(h)})/3$ and the variance ${{\sigma}_{{\eta}}^{2}}=({D-{D_{inf}}(h)})/(3{{\chi}_{z}})$ for the AWGN, there exists ${K_{1}}\in{\kappa}$ rendering the NCS of Fig. 2 internally stable and well-posed in a way that ${{\sigma}_{z^{\prime}}^{2}{\mid}_{(B,J,{\sigma}_{\eta}^{2})=(K_{1},1,{\sigma}_{\eta}^{2})}}\leq{{D_{inf}(h)}+{\frac{2}{3}}(D-{D_{inf}(h)})}<D$. Then the following can be obtained for the structure of Fig. 2:

[TABLE]

So regarding Jensen’s inequality and concavity of logarithm, it can be deduced that the problem of finding ${{\vartheta}_{u}^{{}^{\prime}}(D)}$ in (13) is feasible for every $D>{D_{inf}}(h)$.

-A3 Feasibility of $\varphi^{\prime}(D)$

Immediately from (32), feasibility of the problem of finding $\varphi^{\prime}(D)$ in (25) is inferred for any $D>{D_{inf}}(h)$.

-B Proof of Theorem IV.1

The coding scheme in Fig. 1 comprises lossy, lossless and reproduction part as depicted in Fig. 8. On the encoder side, (3) can be rewritten as follows:

[TABLE]

in which the side informations belong to well-defined sets, i.e. ${{\eta_{e}}(k)}\in{{{\Theta}_{\mathcal{E}}}(k)}$ and ${{\eta_{O}}(k)}\in{{{\Theta}_{{O}}}(k)}$. Likewise, the decoding scheme in (4) is given by

[TABLE]

where ${{u_{\mathcal{D}}}(k)}={{y_{\mathcal{E}}}(k-h)}$ holds for $k\geq{h}$. In addition, ${{\eta_{d}}(k)}\in{{{\Theta}_{\mathcal{D}}}(k)}$ and ${{{\Theta}_{\mathcal{D}}}(k)}$ is a well-defined set. The set ${{{\Theta}_{{O}}}(k)}$ is defined as ${{{\Theta}_{{O}}}(k)}\triangleq{{{{\Theta}_{\mathcal{E}}}(k)}\cap{{{\Theta}_{\mathcal{D}}}(k)}}$. Regarding (33)-(34), the decoder has access to ${y_{\mathcal{E}}^{k-1}}$ and ${\eta_{O}^{k}}$ when receiving ${y_{\mathcal{E}}(k)}$ at time $k+h$. Accordingly, the rate at each time instant is lower-bounded as follows:

[TABLE]

Furthermore, based on the closed-loop dynamics of the NCS depicted in Fig. 1, its measurement output, $y(k)$, satisfies

[TABLE]

where ${\mathcal{G}_{k}}$ is a linear deterministic mapping. Accordingly, we can derive the following chain of inequalities and identities:

[TABLE]

where (p) is immediately deduced from (35). Founded upon one property of the entropy and ${{{\Theta}_{{I}}}(k)}\triangleq{{{{\Theta}_{\mathcal{E}}}(k)}\cap{{{\Theta}_{\mathcal{D}}}(k)}}$, (q) is yielded. (r) and (s) follow from positiveness of the entropy and definition of mutual information, respectively. Additionally, (t) is given based on one property of mutual information and ${{u_{\mathcal{D}}}(k)}={{y_{\mathcal{E}}}(k-h)}$; $k\geq{h}$. According to invertibility of the decoder, (u) is concluded. As a result of Assumption III.1, (34) and (36), Lemma 4.2 in [11] holds for the system of our interest in Fig. 1. From the second claim of the this Lemma, the Markov chain${{y}^{i+h}-{{u}_{\mathcal{D}}(i+h)}-u(i+h)}$, conditioned upon $({{\eta}_{d}^{i+h}},{{u}_{\mathcal{D}}^{i+h-1}})$, is inferred. Since ${{u_{\mathcal{D}}}(i)}={{y_{\mathcal{E}}}(i-h)}$; $i\geq{h}$, The Markov chain ${y}^{i+h}-{{y}_{\mathcal{E}}(i)}-u(i+h)$, conditioned upon $({{\eta}_{d}^{i+h}},{{u}_{\mathcal{D}}^{i+h-1}})$, holds. So the Markov chain ${y}^{i}-{{y}_{\mathcal{E}}(i)}-u(i+h)$, conditioned upon $({{\eta}_{d}^{i+h}},{{u}_{\mathcal{D}}^{i+h-1}})$, holds as well. With all this in mind, the validity of (v) is verified regarding the invertibility of the decoder and (M3) in [11]. (w), (x) and (y) are caused by the chain rule of mutual information. (z) is deduced based upon the Markov chain ${{\eta}_{d}^{i+h}}-{u}^{i+h-1}-{y}^{i}$ which is caused by another Markov chain ${{\eta}_{d}^{i+h}}-{u}^{i+h-1}-{y}^{i+h}$ (Lemma 4.2 in [11]). Hence,

[TABLE]

where (aa) holds due to the positive-ness of mutual information and (ab) stems from Assumption III.2. Consequently

[TABLE]

from which (6) is resulted immediately.