On Instability of LS-based Self-tuning Systems with Bounded Disturbances

TL;DR

This paper demonstrates that least-squares based self-tuning regulators for discrete-time linear systems can become unstable under any bounded disturbance, highlighting a fundamental limitation of these controllers.

Contribution

It reveals the inherent instability risk of LS-based self-tuning regulators in the presence of bounded disturbances, even if the noise is minimal.

Findings

LS-based STR can destabilize systems with bounded disturbances

Instability risk exists regardless of disturbance magnitude

Highlights limitations of LS-based self-tuning control methods

Abstract

It is well known that discrete-time linear systems can be stabilized by a least-squares (LS) based self-tuning regulator (STR), as long as noises are absent. However, this note shows that once the discrete-time linear systems are disturbed, the LS-based STR is always running the risk of unstabilizing systems, no matter how small the noises are.

Equations

$$A(q^{-1})y_{t+1}=B(q^{-1})u_t$$

$$y_{t+1}={\theta }^T{f}_t(y_t,\dots ,y_{t-p+1},u_{t-1},\dots ,u_{t-q}),$$

$$A(q^{-1})y_{t+1}=B(q^{-1})u_t+C(q^{-1})w_{t+1}.$$

$$y_{t+1}=\theta y_t+u_t+w_{t+1},t\ge 0,$$

$${\theta }_{t+1}={\theta }_t+\frac{1}{r_t}{y}_t(y_{t+1}-u_t-{\theta }_t{y}_t),$$

$$\frac{1}{r_t}=\frac{1}{r_{t-1}}-\frac{y_t^2}{r_{t-1}^2+r_{t-1}{y}_t^2},r_0>0,$$

$$u_t=-{\theta }_t{y}_t.$$

$$y_{t+1}=\widetilde{{\theta }_t}{y}_t+w_{t+1}.$$

$$\underset{t\to +\infty }{\overline{\lim }}\frac{\sum _{i=1}^t{y}_i^2}{\sum _{i=1}^t{w}_i^2}=+\infty .$$

$$r_{k-1}\ge w^2,|{\tilde{\theta }}_{k-1}|\le 1,|y_{k-1}|\le \frac{w}{2\sqrt{k-1}}.$$

$$w_t=-{\tilde{\theta }}_{t-1}{y}_{t-1}+\mathcal{S}(y_{t-1})\frac{w}{2\sqrt{t}},t\ge k,$$

$$\mathcal{S}(x)\triangleq\left\{\begin{array}[]{ll}-1,&\mbox{if}~{}x<0\\ 1,&\mbox{if}~{}x\geq 0\end{array}.\right.$$

$${\tilde{\theta }}_{t+1}=\widetilde{{\theta }_t}-\frac{y_t{y}_{y+1}}{r_t},$$

$$y_t=\mathcal{S}(y_{t-1})\frac{w}{2\sqrt{t}},t\ge k,$$

$$|w_k|$$

$$\le \frac{w}{2}+\frac{w}{2}=w.$$

$$|{\tilde{\theta }}_k|$$

$$\le |{\tilde{\theta }}_{k-1}|+|\frac{y_{k-1}{y}_k}{w^2}|$$

$$\le 1+\frac{1}{4}=\frac{5}{4},$$

$$|w_{k+1}|\le |{\tilde{\theta }}_k{y}_k|+|y_{k+1}|\le \frac{5}{4}\frac{w}{2\sqrt{2}}+\frac{w}{2}\le w.$$

$$r_{t-1}\ge r_{k-1}\ge w^2,$$

$$|\widetilde{{\theta }_t}{y}_t|-|{\tilde{\theta }}_{t-1}{y}_{t-1}|$$

$$\le |\frac{y_{t-1}{y}_t^2}{r_{t-1}}|\le \frac{w^3}{8r_{t-1}\sqrt{t-1}\sqrt{t}\sqrt{t}}$$

$$\le \frac{w}{8\sqrt{t-1}\sqrt{t}\sqrt{t}}$$

$$|y_t|-|y_{t+1}|$$

$$=\frac{w}{2\sqrt{t}\sqrt{t+1}(\sqrt{t}+\sqrt{t+1})}.$$

$$4\sqrt{t-1}\sqrt{t}=4\sqrt{2}\ge \sqrt{3}(\sqrt{2}+\sqrt{3})=\sqrt{t+1}(\sqrt{t}+\sqrt{t+1}).$$

$$4\sqrt{t-1}\sqrt{t}$$

$$\ge 2\sqrt{t+1}\sqrt{t+1}$$

$$\ge \sqrt{t+1}(\sqrt{t}+\sqrt{t+1}).$$

$$|\widetilde{{\theta }_t}{y}_t|-|{\tilde{\theta }}_{t-1}{y}_{t-1}|\le |y_t|-|y_{t+1}|,$$

$$|\widetilde{{\theta }_t}{y}_t|+|y_{t+1}|\le |{\tilde{\theta }}_{t-1}{y}_{t-1}|+|y_t|.$$

$$|w_t|$$

$$\le |{\tilde{\theta }}_k{y}_k|+|y_{k+1}|=w.$$

$$r_{t+1}=r_k+\sum _{i=k+1}^{+\infty }{y}_i^2=r_k+\frac{w^2}{4}\sum _{i=k+1}^{+\infty }\frac{1}{i}\to +\infty .$$

Full text

Keywords: Least-squares, self-tuning regulators, parametrization, linear systems, discrete-time, instability, noises

1 Introduction

It was proved early in [2] that the following noise-free system

[TABLE]

can be stabilized by a least-squares based self-tuning regulator. In fact, as long as the noise is absent, the LS-based STR even is capable of stabilizing the nonlinear discrete-time system

[TABLE]

whenever $\|f_{t}(x)\|=O(1)+O(\|x\|^{b})$ with $b<8$ (see [3]). But what will happen if systems are disturbed by bounded noises? This may be more practical. Relevant works in the stochastic framework shed some light (e.g., [4], [5], [6], [7], [8]). [8] studied the ARMA model, which is corrupted by a sequence of martingale difference noises, and derived the stability and optimality of the LS-based STR. Meanwhile, the strong consistency of the LS estimator is guaranteed in the closed loop. Later on, number $b=4$ and a polynomial criterion have been put forward as the critical nonlinear characterizations of the stabilizability for systems in type (1) but with random noises involved (see [9] and [10]). Such systems can be stabilized by the LS-based STR, when their nonlinearities are within the critical nonlinear conditions. Otherwise, no feedback control law is possible to stabilize them. This suggests that noises play an role here. The critical nonlinear growth rates are apparently reduced by the involvement of noises.

Now, a direct consequence of [7] indicates that if the noises are assumed to be bounded and i.i.d distributed, then with probability $1$, the LS-based STR can stabilize system

[TABLE]

The trouble is, there still exist some sequences $\{w_{t}\}$ with probability [math] such that the stochastic tools could do nothing to them. [11] observed the divergence of the LS estimator in a self-tuning system for some special bounded noises. Nevertheless, whether the LS-based self-tuning system is stable or not for bounded noises was still unknown to people yet. We prove in this note that there indeed exist some bounded noises that will result in the instability of a LS-based self-tuning system, even for the simplest discrete-time linear model with a scalar unknown parameter. Perhaps more surprisingly, as our result indicates, once a discrete-time system is disturbed, the LS-based STR is always running the risk of unstabilizing it, no matter how small the noises are. In the meantime, for the bounded noises causing the system unstable, the LS estimator is proved to be divergent during the closed-loop identification, as observed in [11].

Notably, though, discrete-time uncertain systems with bounded noises are stabilizable as well, provided their nonlinearities meet the polynomial criterion (see [12] and [13]). This means, different from the stochastic framework where the LS-STR converges to the minimal variance controller, the LS-STR in the deterministic framework performs no longer “optimal”.

2 Main Results

Consider the discrete-time single-input/single-output linear model:

[TABLE]

where $y_{t},u_{t},w_{t}\in\mathbb{R}$ are the system output, input, and noise sequences, respectively. Parameter $\theta\in\mathbb{R}$ is unknown. Further, we assume

Assumption 1**.**

There is a number $w>0$ such that $|w_{t}|\leq w$ for all $t\geq 1$.

The standard LS estimate $\theta_{t}$ of $\theta$ for model (2) reduces to

[TABLE]

where $\theta_{0},r_{0}$ are the deterministic initial values of the algorithm. The feedback law is designed according to the well-known certainty equivalence principle:

[TABLE]

Denote $\tilde{\theta}_{t}\triangleq\theta-\theta_{t}$, then the closed-loop system (2) and (5) equals to

[TABLE]

Theorem 1**.**

*For any initial values $y_{0}$ and $\theta_{0}$, there exists a sequence $\{w_{t}\}$ satisfying Assumption 1 such that

(i) the LS estimate error $\tilde{\theta}_{t}$ diverges that $\varlimsup_{t\to+\infty}|\tilde{\theta_{t}}|=+\infty$;

(ii) the outputs of the closed-loop system (6) satisfies*

[TABLE]

3 Proof of Theorem 1

The proof is based on several lemmas.

Lemma 1**.**

Let $k\geq 2$ be an integer that

[TABLE]

Then, $|w_{t}|\leq w$ for all $t\geq k$, if

[TABLE]

where

[TABLE]

Proof.

First of all, note that (3) and (4) yield

[TABLE]

and $r_{t}=\sum\limits_{i=1}^{t}{y_{i}^{2}}$ with $r_{0}=y_{0}^{2}$. According to (6) and (9), we have

[TABLE]

We now verify that both $w_{k}$ and $w_{k+1}$ are bounded by $w$. As a matter of fact, by virtue of (6), (8) and (11),

[TABLE]

Moreover, (10) yields

[TABLE]

which by (6) again,

[TABLE]

When $t\geq k+1$, $|y_{t}|\leq|y_{t-1}|$ due to (11). In addition, (8) means

[TABLE]

so, by (10) and (11),

[TABLE]

and

[TABLE]

If $t=2$, we have

[TABLE]

For $t\geq 3$, it is also easy to compute

[TABLE]

Since integer $t\geq 2$, (3) and (3) shows

[TABLE]

which yields

[TABLE]

As a consequence, by (12), we have

[TABLE]

The proof is completed. $\Box$

Lemma 2**.**

Let $k\geq 2$ be an integer fulfilling (8). If $\{w_{t}\}$ satisfies (9), then there is an integer $k^{\prime}\geq k$ such that $\{|\tilde{\theta_{t}}|,t\geq k^{\prime}\}$ is a strictly increasing sequence and $\lim_{t\to+\infty}|\tilde{\theta_{t}}|=+\infty$.

Proof.

When $\{w_{t}\}$ satisfies (9), (11) holds and as $t\to+\infty$,

[TABLE]

So, as $t\to+\infty$,

[TABLE]

which immediately shows

[TABLE]

Now, since $y_{t}y_{t+1}\geq 0$ for all $t\geq k+1$, by (11),

[TABLE]

In view of (10), $\tilde{\theta}_{t}$ decreases monotonically and $\lim_{t\to+\infty}\theta_{t}=-\infty.$ Let $k^{\prime}\triangleq\min\{t:\tilde{\theta}_{t}<0\},$ then $\{|\tilde{\theta}_{t}|,t\geq k^{\prime}\}$ is a strictly increasing sequence. $\Box$

Lemma 3**.**

*Let $j\geq 0$ be an integer that $y_{j}\neq 0,~{}\tilde{\theta}_{j}\neq 0$ and $w_{t}=0$ for all $t\geq j+1$. The following two statements hold:

(i) there are some numbers $c_{0}>0$, $\alpha\in(0,1)$ and $t_{0}\geq j$ such that*

[TABLE]

(ii) if $|\tilde{\theta}_{j}|\geq 6$, then there is an integer $l\geq j$ such that

[TABLE]

Proof.

We now prove statement (i). By (6) and (10),

[TABLE]

From (6), (10) and (16), for all $t\geq j$,

[TABLE]

Since $\{r_{t}\}$ is an increasing sequence, (18) shows that $\{|\tilde{\theta}_{t}|,t\geq j\}$ is a decreasing sequence. So, $|\tilde{\theta}_{t}|$ is bounded for all $t\geq j$ and $\lim_{t\to+\infty}|\tilde{\theta}_{t}|$ exists. We assert $\lim_{t\to+\infty}|\tilde{\theta}_{t}|<1$. Otherwise, if $\lim_{t\to+\infty}|\tilde{\theta}_{t}|\geq 1$, then $|\tilde{\theta}_{t}|\geq 1$ for all $j\geq t$. By (17), we have

[TABLE]

This immediately leads to $\lim_{t\to+\infty}r_{t}=+\infty$. By $\lim_{t\to+\infty}|\tilde{\theta}_{t}|\geq 1$ again,

[TABLE]

which contradicts to (19). Therefore, $\lim_{t\to+\infty}|\tilde{\theta}_{t}|<1$ and hence, $|\tilde{\theta}_{t_{0}}|<1$ for some integer $t_{0}\geq j$.

When $t\geq t_{0}\geq j$, by the fact that $\{|\tilde{\theta}_{t}|,t\geq j\}$ is a decreasing sequence, (17) shows that

[TABLE]

The first formula of (i) is thus derived by letting $c_{0}=|y_{t_{0}}||\tilde{\theta}_{t_{0}}|^{-{t_{0}}}$ and $\alpha=|\tilde{\theta}_{t_{0}}|\in(0,1)$. So, as $t\to+\infty$,

[TABLE]

This together with (19) infers that $\lim_{t\to+\infty}|\tilde{\theta}_{t}|>0$.

To prove statement (ii), we first assert that there is an integer $l\geq j$ that $r_{l}\geq 3r_{j-1}$. Otherwise, $r_{t-1}<3r_{j-1}$ for all $t\geq j$. Therefore, by $|\tilde{\theta}_{j}|\geq 6$ and (19), we have for any $t\geq j$,

[TABLE]

Consequently, by (20), as $t\to+\infty$,

[TABLE]

which contradicts to (21). Let

[TABLE]

then $r_{l-1}\leq 3r_{j-1}$ and $r_{l}\geq 3r_{j-1}$.

The remainder is devoted to showing $y_{l}^{2}\geq r_{j-1}$. Otherwise, if $y_{l}^{2}<r_{j-1}$, by noting that $r_{t-1}\leq 3r_{j-1}$ for any $j+1\leq t\leq l$, (22) holds as well for all $t\in[j+1,l]$ and hence $\frac{1}{|\tilde{\theta}_{t}|}<\frac{1}{2}$. Rewrite $r_{t}$ by

[TABLE]

which derives a contradiction. (ii) is thus proved. $\Box$

Lemma 4**.**

Let $k\geq 2$ satisfy (8). Given a constant $c>0$, set

[TABLE]

where $j\geq k$ is an integer such that

[TABLE]

Then, $(\sum_{i=1}^{+\infty}y_{i}^{2})/(\sum_{i=1}^{+\infty}w_{i}^{2})\geq c.$

Proof.

By (6), (8), (23) and (24), we have

[TABLE]

Note that $w_{t}=0$ for all $t\geq j+1$, in view of Lemma 3, there exists an integer $l\geq j$ fulfilling (15). By (19), $r_{l-1}\leq 3r_{j-1}$ yields $|\tilde{\theta}_{l}|^{2}\geq\frac{1}{9}|\tilde{\theta}_{j}|^{2}$. Then, (15), (17) and (19) imply

[TABLE]

Moreover, $|\tilde{\theta}_{j}|\geq 126c$, the above inequality shows

[TABLE]

which completes the proof. $\Box$

Proof of Theorem 1.

Set the noise

[TABLE]

where for $\mathcal{K}_{1}\triangleq\{k\geq 3:(\ref{k})\mbox{ holds}\}$,

[TABLE]

Clearly, $|w_{t}|\leq w$ for $t\in[1,k_{1}-1]$. We next claim that $k_{1}$ is finite. Otherwise, $\mathcal{K}_{1}=\emptyset$. Then, (8) fails for every $k\geq 3$ and $w_{t}=0$ whenever $t\geq 2$. Now, $w_{1}=\mathcal{S}(\tilde{\theta}_{0}y_{0})w$, which means $|w_{1}|=w$ and $y_{1}^{2}\geq w^{2}>0$. Therefore,

[TABLE]

If $\tilde{\theta}_{1}=0$, it is easy to compute that $\tilde{\theta}_{2}=y_{2}=0$ due to $w_{2}=0$. So, (8) holds for $k=3$. This asserts $\tilde{\theta}_{1}\neq 0$. Consequently, by Lemma 3(i), there are some $c_{0}>0$ and $\alpha\in(0,1)$ such that for all sufficiently large $k\geq 3$,

[TABLE]

which together with (26) contradicts to $\mathcal{K}_{1}=\emptyset$. Therefore, $k_{1}$ is finite.

Now, fix $k_{1}\in\mathcal{K}_{1}$. Define

[TABLE]

where

[TABLE]

Moreover, let

[TABLE]

with

[TABLE]

For $j_{1}$ and $k_{2}$ defined above, set

[TABLE]

Noting that (8) is true for $k=k_{1}$, by Lemma 1, $|w_{t}|\leq w$ for all $t\in[k_{1},k_{2}-1].$

We proceed to prove that both $j_{1}$ and $k_{2}$ are finite. If $j_{1}=+\infty$, then $w_{t}$ satisfies (9) for all $t\geq k_{1}$. Further, since (8) holds for $k=k_{1}$, by Lemma 2, $\{|\tilde{\theta}_{t}|,t\geq k^{\prime}_{1}\}$ is an increasing sequence for some $k^{\prime}_{1}\geq k_{1}$ and $\lim_{t\to+\infty}|\tilde{\theta_{t}}|=+\infty$, which gives $\mathcal{J}_{1}\neq\emptyset$. Hence $j_{1}$ is finite or a contradiction arises. So, by Lemma 4, we immediately deduce that for all sufficiently large $k$,

[TABLE]

It is clear that $\tilde{\theta}_{j_{1}}\neq 0$ as $j_{1}\in\mathcal{J}_{1}$ and $y_{j_{1}}\neq 0$ by (11). Similar to the proof of $k_{1}<+\infty$, Lemma 3(i) shows that $k_{2}$ is finite.

Suppose two increasing sequences $\{k_{i}\in\mathbb{N}^{+},1\leq i\leq s\}$, $\{j_{i}\in[k_{i},k_{i+1}-2]\cap\mathbb{N}^{+},1\leq i\leq s-1\}$ and a series $\{w_{t},1\leq t\leq k_{s}-1\}$ are constructed for some $s\geq 2$ such that (8) holds for $k=k_{s}$, $|\tilde{\theta}_{j_{s-1}}|\geq 126(s-1)$ and

[TABLE]

Analogous arguments of (27) and (28)–(29) yield that there are two finite integers $k_{s+1}$ and $j_{s}$, as well as a sequence $\{w_{t},k_{s}\leq t\leq k_{s+1}-1\}$ such that (8) holds for $k=k_{s+1}$, $|\tilde{\theta}_{j_{s}}|\geq 126s$ and

[TABLE]

So, there exists a $\{w_{t},t\geq 1\}$ and a $\{(k_{s},j_{s}),s\geq 1\}$ fulfilling (30) and $|\tilde{\theta}_{j_{s}}|\geq 126s$ for each $s\geq 2$. Statements (i) and (ii) are thus derived as desired. $\Box$