The regular indefinite linear quadratic optimal control problem: stabilizable case

TL;DR

This paper extends the theory of indefinite linear quadratic optimal control to stabilizable systems, providing explicit solutions and conditions for optimal control existence in a more general setting.

Contribution

It generalizes previous controllable case results to stabilizable systems, explicitly characterizing the unique algebraic Riccati equation solution.

Findings

Explicit characterization of the unique Riccati solution.

Necessary and sufficient conditions for optimal control existence.

Extension of control theory to stabilizable systems.

Abstract

This paper addresses an open problem in the area of linear quadratic optimal control. We consider the regular, infinite-horizon, stability-modulo-a-subspace, indefinite linear quadratic problem under the assumption that the dynamics are stabilizable. Our result generalizes previous works dealing with the same problem in the case of controllable dynamics. We explicitly characterize the unique solution of the algebraic Riccati equation that gives the optimal cost and optimal feedback control, as well as necessary and sufficient conditions for the existence of optimal controls.

Equations

$$\dot{x}=Ax+Bu,x(0)=x_0,$$

$$J_T(x_0,u)={\int }_0^T\omega (x(t;x_0,u))dt$$

$$\omega (x,u):=x^{\top }Qx+u^{\top }Ru=\left[\begin{array}{cc}{x}^{\top }& u^{\top }\end{array}\right]W\left[\begin{array}{c}x\\ u\end{array}\right],W:=\left[\begin{array}{cc}Q& 0\\ 0& R\end{array}\right],R=I_m.$$

$$\mathcal{U}(x_0):=\left\{u\in L_{2,loc}^m({\mathbb{R}}^{+})|\underset{T\to \infty }{\lim }{J}_T(x_0,u)\text{ exists in }{\mathbb{R}}^e\right\}.$$

$${\mathcal{U}}_{\mathcal{L}}(x_0):=\left\{u\in \mathcal{U}(x_0)|\underset{t\to \infty }{\lim }{d}_{\mathcal{L}}(x(t;x_0,u))=0\right\}.$$

$$J(x_0,u):=\underset{T\to \infty }{\lim }{J}_T(x_0,u).$$

$$V_{\mathcal{L}}(x_0):=\inf \{J(x_0,u)|u\in {\mathcal{U}}_{\mathcal{L}}(x_0)\}.$$

$$\varphi (K):=A^{\top }K+KA+Q-KB{B}^{\top }K=0.$$

$$A(K):=A-B{B}^{T}K.$$

$$\Gamma$$

$$\partial \Gamma$$

$${\Gamma }_{-}$$

$$\gamma (\mathcal{V}):=K^{-}{P}_{\mathcal{V}}+K^{+}(I_n-P_{\mathcal{V}}),$$

$$x^{\top }Kx=-(Hx{)}^{\top }(Hx)=0\iff Hx=0\Rightarrow -H^{\top }(Hx)=Kx=0.$$

$${\mathbb{R}}^n=\mathcal{C}\oplus {\mathcal{X}}_2.$$

$$A=\left[\begin{array}{cc}{A}_1& A_{12}\\ 0& A_2\end{array}\right],B=\left[\begin{array}{c}{B}_1\\ 0\end{array}\right].$$

$$Q=\left[\begin{array}{cc}{Q}_1& Q_{12}\\ Q_{12}^{\top }& Q_2\end{array}\right],K=\left[\begin{array}{cc}{K}_1& K_{12}\\ K_{12}^{\top }& K_2\end{array}\right],$$

$$\varphi (K)=\left[\begin{array}{cc}{\varphi }_1(K_1)& A_1(K_1{)}^{\top }{K}_{12}+K_{12}{A}_2+K_1{A}_{12}+Q_{12}\\ \ast & A_2^{\top }{K}_2+K_2{A}_2+K_{12}^{\top }{A}_{12}+A_{12}^{\top }{K}_{12}+Q_2-K_{12}^{\top }{B}_1{B}_1^{\top }{K}_{12}\end{array}\right].$$

$$A_1(K_1):=A_1-B_1{B}_1^{\top }{K}_1.$$

$${\varphi }_1(K_1):=A_1^T{K}_1+K_1{A}_1+Q_1-K_1{B}_1{B}_1^{\top }{K}_1$$

$$A_1(K_1{)}^{\top }{K}_{12}+K_{12}{A}_2$$

$$A_2^{\top }{K}_2+K_2{A}_2$$

$${\Gamma }_1$$

$$\partial {\Gamma }_1$$

$${\Gamma }_{1-}$$

$$\partial {\Gamma }_{1-}$$

$${\mathcal{L}}_1$$

$${\mathcal{N}}_1({\mathcal{L}}_1)$$

$${\overline{K}}_1:=\gamma ({\mathcal{N}}_1({\mathcal{L}}_1))=K_1^{-}{P}_{{\mathcal{N}}_1({\mathcal{L}}_1)}+K_1^{+}(I_{n_1}-P_{{\mathcal{N}}_1({\mathcal{L}}_1)}).$$

$${\mathcal{X}}_{1,1}$$

$${\mathcal{X}}_{1,2}$$

$${\mathcal{X}}_{1,3}$$

$${\mathbb{R}}^n={\mathcal{X}}_{1,1}\oplus {\mathcal{X}}_{1,2}\oplus {\mathcal{X}}_{1,3}\oplus {\mathcal{X}}_2.$$

$$A=\left[\begin{array}{cc}{A}_1& A_{12}\\ 0& A_2\end{array}\right]=\left[\begin{array}{cccc}{A}_{1,11}& A_{1,12}& A_{1,13}& A_{12,1}\\ A_{1,21}& A_{1,22}& A_{1,23}& A_{12,2}\\ A_{1,31}& A_{1,32}& A_{1,33}& A_{12,3}\\ 0& 0& 0& A_2\end{array}\right],B=\left[\begin{array}{c}{B}_1\\ 0\end{array}\right]=\left[\begin{array}{c}{B}_{1,1}\\ B_{1,2}\\ B_{1,3}\\ 0\end{array}\right].$$

$$Q=\left[\begin{array}{cc}{Q}_1& Q_{12}\\ Q_{12}^{\top }& Q_2\end{array}\right]=\left[\begin{array}{cccc}{Q}_{1,11}& Q_{1,12}& Q_{1,13}& Q_{12,1}\\ Q_{1,12}^{\top }& Q_{1,22}& Q_{1,23}& Q_{12,2}\\ Q_{1,13}^{\top }& Q_{1,23}^{\top }& Q_{1,33}& Q_{12,3}\\ Q_{12,1}^{\top }& Q_{12,2}^{\top }& Q_{12,3}^{\top }& Q_2\end{array}\right],K=\left[\begin{array}{cc}{K}_1& K_{12}\\ K_{12}^{\top }& K_2\end{array}\right]=\left[\begin{array}{cccc}{K}_{1,11}& K_{1,12}& K_{1,13}& K_{12,1}\\ K_{1,12}^{\top }& K_{1,22}& K_{1,23}& K_{12,2}\\ K_{1,13}^{\top }& K_{1,23}^{\top }& K_{1,33}& K_{12,3}\\ K_{12,1}^{\top }& K_{12,2}^{\top }& K_{12,3}^{\top }& K_2\end{array}\right].$$

Full text

{AMS}

93C05, 93C35

1 Introduction

In this paper we consider the regular, infinite-horizon linear quadratic optimal control problem in which the cost functional is the integral of an indefinite quadratic form. The regular linear quadratic (LQ) problem, when the quadratic form in the cost functional is positive definite in the control variables, has been studied extensively in the literature [3, 22, 2]. It has been especially well studied under the standard assumption, the so-called positive semidefinite case, when the quadratic form in the cost functional is positive semidefinite in the control and state variables simultaneously. The more general indefinite case imposes no definiteness condition in the control and state variables simultaneously [19, 17]. The LQ problem is termed infinite-horizon if the cost functional is integrated over time from zero to infinity. Finally, the most typical treatment of the LQ problem is the fixed-endpoint problem where the state is required to converge to zero as time tends to infinity. The case when no such condition is imposed has also been studied and is referred to as the free-endpoint problem [17, 16, 8]. In fact, an entire family of LQ problems can be obtained by requiring that the state converges to a subspace. This so-called stability-modulo-a-subspace family of LQ problems includes the fixed- and free-endpoint problems as special cases [16, 8]. For the remainder of the paper, we restrict our attention to the regular and infinite-horizon versions of the problem, for otherwise the optimization problem may yield optimal controllers that are not static linear state feedbacks [20, 2]. Also, we focus on stability-modulo-a-subspace, since it is the more general case.

Traditionally, a complete solution of any variant of the LQ problem requires to find necessary and sufficient conditions for the existence of a finite optimal cost and optimal controls. Existence of a finite optimal cost is called well-posedness, while existence of an optimal control is called attainability. Further, when they exist, a complete solution involves determining the optimal cost and an optimal control. Both should be expressed in terms of the given problem data; that is, the system matrices, the instantaneous cost matrices, and the desired subspace.

In the regular, infinite-horizon, fixed-endpoint, positive semidefinite case, the LQ problem was fully resolved in 1968 by Wonham [21, 22], resulting in the well known necessary and sufficient conditions involving stabilizability and detectability. The corresponding free-endpoint LQ problem was fully characterized much later [6, 18], resulting in conditions involving output stabilizability, a condition less strict than stabilizability [6, 18]. In the regular, infinite-horizon, indefinite case, the fixed-endpoint problem was solved in 1971 by Willems [19], while the free-endpoint problem and general stability-modulo-a-subspace were addressed in 1989 by Trentelman [17, 16]. Importantly, all of the indefinite cases made use of the assumption that the dynamics are controllable. Moreover the solutions are incomplete in that only sufficient conditions for the existence of a finite optimal cost were given (except for the fixed-endpoint problem). The main contribution of this paper is to extend the above results for the regular, infinite-horizon, stability-modulo-a-subspace, indefinite case of the LQ problem. Rather than assuming controllability, we only require stabilizability.

It is well known that in both the positive semidefinite and indefinite cases of the regular, infinite-horizon, stability-modulo-a-subspace LQ problem, the optimal cost and optimal controls are given in terms of a particular solution of the algebraic Riccati equation (ARE) [18, 17]. In the treatment of the regular, infinite-horizon, indefinite LQ problem, the controllability assumption is crucial in order to utilize the geometry of the set of all real symmetric solutions of the ARE [19, 11]. In particular, if this solution set is nonempty, there exist a maximal and minimal solution of the ARE [11]. The regular, infinite-horizon, fixed-endpoint LQ problem, both definite and indefinite cases, has always been easier in the sense that the optimal cost and feedback control law are given in terms of the maximal solution, which is the only solution that can stabilize the closed-loop system [19, 21]. For the regular, infinite-horizon, stability-modulo-a-subspace, indefinite case and under the assumption of controllability, the optimal cost and feedback control law are given by a real symmetric solution to the ARE that depends on both its maximal and minimal solutions [16]. In contrast, under the stabilizability assumption, it is unclear which solution of the ARE to select because the geometry of the set of all real symmetric ARE solutions is less well-behaved. In particular, the minimal solution may no longer exist [10, 11]. This ambiguity of the correct choice of ARE solution for the regular, infinite-horizon, stability-modulo-a-subspace, indefinite LQ problem under merely stabilizable dynamics was discussed by Geerts [7, 8], but it has remained elusive.

In this paper we give the exact form of the optimal feedback that solves the regular, infinite-horizon, stability-modulo-a-subspace, indefinite LQ problem under stabilizable dynamics. Thus we resolve the ambiguity regarding which solution of the ARE to take. Our result requires two assumptions, which are precisely our sufficient conditions for well-posedness: existence of a negative semidefinite solution to the algebraic Riccati inequality (ARI) and stabilizability of the system dynamics. These assumptions may be compared to the sufficient conditions for well-posedness in [17]: existence of a negative semidefinite solution to the ARE and controllability of the system dynamics. The first assumption on existence of a negative semidefinite solution of the ARE or ARI provides for a lower bound on the value function, based on a result of Molinari [12]. Our generalization to the ARI is based on an observation by Geerts [7]. The generalization to the case when the dynamics are stabilizable proves to be the more difficult challenge, as discussed above. This extension constitutes the central contribution of the paper. Finally, we give necessary and sufficient conditions for optimal controls to exist, which, as pointed out in [17], are nontrivial for regular, infinite-horizon, non-fixed-endpoint, indefinite LQ problems.

As a further validation of the correctness of our results, we recover known results for other variants of the regular, infinite-horizon LQ problem by adding assumptions to match those problems. In the regular, infinite-horizon, stability-modulo-a-subspace, indefinite case, if we assume controllable dynamics, we obtain the same necessary and sufficient conditions for the existence of optimal controls, the same form of the optimal cost, and the same form of the optimal control as stated in [19, 17, 16]. In the regular, infinite-horizon, positive semidefinite LQ problem, for both the fixed- and free-endpoint cases, if we assume positive semidefineness, then we again obtain the same necessary and sufficient conditions for the existence of optimal controls, the same form of the optimal cost, and the same form of the optimal control as stated in [18].

Our resolution of the gap in the LQ literature provides more than just an answer to an academic question. Recently, the work in [13] considered a linear term in the state of the cost functional and a free-endpoint objective, albeit over the finite-horizon; with a transformation, this cost can be converted to an indefinite problem with stabilizable but not controllable dynamics. The gap was also recently discussed in [4], which deals with the cooperative indefinite LQ problem. As such, our result has application to game theoretic formulations and economics.

The outline of this paper is as follows. In the remainder of this section we will introduce most of the notational conventions that will be used. In Section 2 we present the problem statement. In Section 3 we summarize the key ingredients needed regarding the geometry of the ARE solutions. In Section 4 we state and prove our main results. In Section 5 we compare our main result to existing results in the literature.

Notation.  We use the following notation. Let $I_{n}$ be the $n\times n$ identity matrix (the subscript is omitted if the dimension is clear from the context). Let $P^{\dagger}$ denote the (unique) pseudo-inverse of $P\in{\mathbb{R}}^{n\times m}$. The set of eigenvalues of $A\in{\mathbb{R}}^{n\times n}$ is denoted by $\sigma(A)$. A subspace ${\mathcal{V}}\subset{\mathbb{R}}^{n}$ is $A-$invariant if $A\mathcal{V}\subset\mathcal{V}$. We use the following subsets of the complex plane: ${\mathbb{C}}^{-}:=\{s\in{\mathbb{C}}\;|\;\textup{Re}(s)<0\}$, ${\mathbb{C}}^{0}:=\{s\in{\mathbb{C}}\;|\;\textup{Re}(s)=0\}$, and ${\mathbb{C}}^{+}:=\{s\in{\mathbb{C}}\;|\;\textup{Re}(s)>0\}$. Given a real monic polynomial $p$ there is a unique factorization $p=p_{-}\cdot p_{0}\cdot p_{+}$ into real monic polynomials with $p_{-}$, $p_{0}$, and $p_{+}$ having all roots in ${\mathbb{C}}^{-}$, ${\mathbb{C}}^{0}$, and ${\mathbb{C}}^{+}$, respectively. Then if $A\in{\mathbb{R}}^{n\times n}$ and if $p$ is its characteristic polynomial, then we define the spectral subspaces $\mathcal{X}^{-}(A):=\textup{Ker}(p_{-}(A))$, $\mathcal{X}^{0}(A):=\textup{Ker}(p_{0}(A))$, and $\mathcal{X}^{+}(A):=\textup{Ker}(p_{+}(A))$. Each of these subspaces are $A-$invariant and the restriction of $A$ to $\mathcal{X}^{-}(A)(\mathcal{X}^{0}(A),\mathcal{X}^{+}(A))$ has characteristic polynomial $p_{-}(p_{0},p_{+})$. For two subspaces ${\mathcal{V}}$ and ${\mathcal{W}}$, let ${\mathcal{V}}\oplus{\mathcal{W}}$ denote their direct sum and let ${\mathcal{V}}\sim{\mathcal{W}}$ denote that they are isomorphic. For an arbitrary matrix $A\in{\mathbb{R}}^{n\times n}$ and subspace ${\mathcal{V}}\subset{\mathbb{R}}^{n}$ we define the subspace $\langle A\;|\;{\mathcal{V}}\rangle:={\mathcal{V}}+A{\mathcal{V}}+\ldots A^{n-1}{\mathcal{V}}$, and by further writing ${\mathcal{V}}=\textup{Ker}(W)$ for some $W\in{\mathbb{R}}^{p\times n}$ we also define $\langle{\mathcal{V}}\;|\;A\rangle:=\textup{Ker}(W)\cap\textup{Ker}(WA)\ldots\cap\textup{Ker}(WA^{n-1})$. For a linear time-invariant system, $\dot{x}=Ax+Bu$, the controllable subspace will be denoted in the usual way $\langle A\;|\;\textup{Im}(B)\rangle$. If there is an output $y=Cx$, then $\langle\textup{Ker}(C)\;|\;A\rangle$ denotes the unobservable subspace of $(C,A)$. If $M$ is a real $n\times n$ matrix and ${\mathcal{V}}$ is a subspace of ${\mathbb{R}}^{n}$, then $M^{-1}{\mathcal{V}}:=\{x\in{\mathbb{R}}^{n}\;|\;Mx\in{\mathcal{V}}\}$. If ${\mathcal{V}}$ is a subspace of ${\mathbb{R}}^{n}$ then ${\mathcal{V}}^{\perp}$ denotes its orthogonal complement with respect to the standard Euclidean inner product.

Let ${\mathbb{R}}^{+}:=\{t\in{\mathbb{R}}\;|\;t\geq 0\}$ and ${\mathbb{R}}^{e}:={\mathbb{R}}\cup\{-\infty,+\infty\}$. Additionally, given a function $f:{\mathbb{R}}\rightarrow{\mathbb{R}}$, the statement that $\lim_{t\rightarrow\infty}f(t)$ exists in ${\mathbb{R}}^{e}$ means that $\lim_{t\rightarrow\infty}f(t)$ is either equal to a real number, $\infty$, or $-\infty$ in the usual sense.

We denote the space of all measurable vector-valued functions on ${\mathbb{R}}^{+}$ that are locally square integrable as $L_{2,loc}^{m}({\mathbb{R}}^{+})=\left\{\left.u:{\mathbb{R}}^{+}\rightarrow{\mathbb{R}}^{m}\;\right|\;(\forall T\geq 0)\int_{0}^{T}u(t)^{\top}u(t)\,dt<\infty\right\}$. Let $d_{{\mathcal{L}}}:{\mathbb{R}}^{n}\rightarrow[0,\infty)$ denote the function giving the minimum Euclidean distance from a point to a set $\mathcal{L}\subset{\mathbb{R}}^{n}$.

Given a quadratic form on ${\mathbb{R}}^{n}$, $\omega:{\mathbb{R}}^{n}\rightarrow{\mathbb{R}}$, it is said to be positive definite if for all $x\in{\mathbb{R}}^{n}$, $\omega(x)\geq 0$, and $\omega(x)=0$ if and only if $x=0$; positive semidefinite if for all $x\in{\mathbb{R}}^{n}$, $\omega(x)\geq 0$; negative definite if $-\omega$ is positive definite; negative semidefinite if $-\omega$ is positive semidefinite; and indefinite if $\omega$ is neither positive semidefinite nor negative semidefinite. Writing $\omega(x):=x^{\top}Px$ for some symmetric matrix $P\in{\mathbb{R}}^{n\times n}$, we say that the matrix $P$ is positive definite if the quadratic form $\omega$ is positive definite and so on. We write $P>0$, $P\geq 0$, $P<0$, and $P\leq 0$ if the matrix is positive definite, positive semidefinite, negative definite, and negative semidefinite, respectively. Given symmetric matrices $P,Q\in{\mathbb{R}}^{n\times n}$, we write $P<Q$ if $Q-P>0$, and likewise for the other inequalities. Let $\Lambda$ denote a subset of the set of all symmetric matrices in ${\mathbb{R}}^{n\times n}$. We say that $M^{+}$* ($M^{-}$) is the maximal (minimal) element on $\Lambda$* if $M^{+}\in\Lambda$ ($M^{-}\in\Lambda$) and for all $M\in\Lambda$, $M\leq M^{+}$ ($M\geq M^{-}$). The maximal and minimal elements, which are called the extremal elements on $\Lambda$, are unique if they exist since $\Lambda$ forms a partially ordered set.

2 Problem Statement

We consider the linear control system

[TABLE]

where $x\in{\mathbb{R}}^{n}$ and $u\in{\mathbb{R}}^{m}$. For a control function $u\in L_{2,loc}^{m}({\mathbb{R}}^{+})$, let $x(\cdot;x_{0},u)$ denote the state trajectory of (1) starting at $x_{0}\in{\mathbb{R}}^{n}$. Then for $T\geq 0$, the cost function is

[TABLE]

with a quadratic instantaneous cost

[TABLE]

We allow $Q$ to be indefinite, whereas $R:=I_{m}>0$. More general quadratic cost functions can be considered, but they can be converted via a feedback transformation to the form we use here, as in Chapter 10 of [18]. This feedback transformation does not affect solvability of the problem; hence, there is no loss of generality in our choice of $W$.

Because $W$ may be indefinite, we define the set of control inputs that yield a cost that is either finite, $\infty$, or $-\infty$:

[TABLE]

Let $\mathcal{L}\subset{\mathbb{R}}^{n}$ be a subspace. The set of permissible control inputs such that the state asymptotically converges to ${\mathcal{L}}$ is

[TABLE]

For $u\in{\mathcal{U}}_{{\mathcal{L}}}(x_{0})$, we define

[TABLE]

We define the optimal cost or value function to be

[TABLE]

Now we define the linear quadratic optimal control problem with stability-modulo-${\mathcal{L}}$ $\text{(LQCP)}_{\mathcal{L}}~{}$.

Problem 2.1** ($\text{(LQCP)}_{\mathcal{L}}~{}$).**

*Consider the system (1) with the quadratic cost criterion (2). Let $\mathcal{L}\subset{\mathbb{R}}^{n}$ be a given subspace. For all $x_{0}\in{\mathbb{R}}^{n}$, find the optimal cost $V_{{\mathcal{L}}}(x_{0})$ and an optimal control $u^{\star}\in{\mathcal{U}}_{{\mathcal{L}}}(x_{0})$ such that $V_{{\mathcal{L}}}(x_{0})=J(x_{0},u^{\star})$. *

The $\text{(LQCP)}_{\mathcal{L}}~{}$is called regular (as opposed to singular) if $R>0$. It is called positive semidefinite if $\omega$ is positive semidefinite on ${\mathbb{R}}^{n+m}$, and indefinite otherwise. If $\mathcal{L}={\mathbb{R}}^{n}$, the $\text{(LQCP)}_{\mathcal{L}}~{}$is called a free-endpoint problem, and if $\mathcal{L}=0$, it is called a fixed-endpoint problem. We are particularly interested in characterizing two properties of the $\text{(LQCP)}_{\mathcal{L}}~{}$.

Definition 1**.**

*We say the $\text{(LQCP)}_{\mathcal{L}}~{}$is well-posed if for all $x_{0}\in{\mathbb{R}}^{n}$, $V_{\mathcal{L}}(x_{0})\in{\mathbb{R}}$. We say the $\text{(LQCP)}_{\mathcal{L}}~{}$is attainable if for all $x_{0}\in{\mathbb{R}}^{n}$, there exists a control $u^{\star}\in{\mathcal{U}}_{{\mathcal{L}}}(x_{0})$ such that $V_{\mathcal{L}}(x_{0})=J(x_{0},u^{\star})$. Such an input is called optimal. We say the $\text{(LQCP)}_{\mathcal{L}}~{}$is solvable if it is both well-posed and attainable. *

3 Preliminaries

The main results on the $\text{(LQCP)}_{\mathcal{L}}~{}$are centered on the algebraic Riccati equation (ARE):

[TABLE]

The algebraic Riccati inequality (ARI) is given by $\phi(K)\geq 0$. For convenience, we define

[TABLE]

Also we define the following solution sets:

[TABLE]

The geometry of the solutions to the ARE can be studied in both the controllable and stabilizable cases; see, in particular, Chapters 7 and 8 of [11] and also [17]. First we consider the case when $(A,B)$ is controllable. The next result summarizes what is known about the extremal solutions in $\Gamma$ and in $\partial\Gamma$.

Theorem 3.1**.**

Suppose $(A,B)$ is controllable.

• (i)

If $\Gamma\neq\emptyset$, then the maximal and minimal solutions in $\Gamma$ exist, $\partial\Gamma\neq\emptyset$, its maximal and minimal solutions exist, and they are identical to the maximal and minimal solutions in $\Gamma$.

• (ii)

If $\partial\Gamma\neq\emptyset$, then its maximal and minimal solutions $K^{+},K^{-}\in\partial\Gamma$ satisfy: $\forall K\in\partial\Gamma$, $K^{-}\leq K\leq K^{+}$. Moreover, they are the unique solutions in $\partial\Gamma$ such that $\sigma(A(K^{+}))\subset{\mathbb{C}}^{-}\cup{\mathbb{C}}^{0}$ and $\sigma(A(K^{-}))\subset{\mathbb{C}}^{+}\cup{\mathbb{C}}^{0}$.

Proof 3.2**.**

*The first statement is Theorem 14(b) in [14]. The second statement was proved in [19]. See also Theorem 7.5.1, p. 168, in [11]. *

If $\partial\Gamma\neq\emptyset$, define the gap of the ARE to be $\Delta:=K^{+}-K^{-}$. Let $\Omega$ denote the set of all $A(K^{-})-$invariant subspaces contained in $\mathcal{X}^{+}(A(K^{-}))$. The following theorem was first proven by Willems [19]; see also [11].

Theorem 3.3** (Theorem 3.1, [17]).**

Let $(A,B)$ be controllable and suppose $\partial\Gamma\neq\emptyset$. If $\mathcal{V}\subset\Omega$, then ${\mathbb{R}}^{n}=\mathcal{V}\oplus\Delta^{-1}(\mathcal{V}^{\perp})$. There exists a bijection $\gamma:\Omega\rightarrow\partial\Gamma$ defined by

[TABLE]

*where $P_{\mathcal{V}}$ is the projection onto $\mathcal{V}$ along $\Delta^{-1}(\mathcal{V}^{\perp})$. If $K=\gamma(\mathcal{V})$, then $\mathcal{X}^{+}(A(K))=\mathcal{V}$, $\mathcal{X}^{0}(A(K))=\textup{Ker}(\Delta)$, and $\mathcal{X}^{-}(A(K))=\mathcal{X}^{-}(A(K^{+}))\cap\Delta^{-1}(\mathcal{V}^{\perp})$. *

An application of Theorem 3.3 is the main result of [16], which provides a solution of the $\text{(LQCP)}_{\mathcal{L}}~{}$when $(A,B)$ is controllable. To state the sufficient condition for well-posedness, an additional definition is needed from [16]: for a given subspace ${\mathcal{L}}\subset{\mathbb{R}}^{n}$ and symmetric matrix $K\in{\mathbb{R}}^{n\times n}$, $K$ is said to be negative semidefinite on ${\mathcal{L}}$ if for all $x\in{\mathcal{L}}$, $x^{\top}Kx\leq 0$, and $x^{\top}Kx=0$ if and only if $Kx=0$. Notice that $K\leq 0$ implies that for all ${\mathcal{L}}\subset{\mathbb{R}}^{n}$, $K$ is negative semidefinite on ${\mathcal{L}}$. To see this, fix ${\mathcal{L}}\subset{\mathbb{R}}^{n}$ and note that $K\leq 0$ implies that there exists $H\in{\mathbb{R}}^{p\times n}$ for some $p$ such that $K=-H^{\top}H$. Then for all $x\in{\mathcal{L}}\subset{\mathbb{R}}^{n}$, obviously $x^{\top}Kx\leq 0$, $Kx=0$ implies $x^{\top}Kx=0$, and

[TABLE]

Theorem 3.4** (Theorem 4.1, [16]).**

Let $(A,B)$ be controllable. Assume $\partial\Gamma\neq\emptyset$ and $K^{-}$ is negative semidefinite on ${\mathcal{L}}$. Then we have

1. (i)

For all $x_{0}\in{\mathbb{R}}^{n}$, $V_{{\mathcal{L}}}(x_{0})$ is finite. 2. (ii)

For all $x_{0}\in{\mathbb{R}}^{n}$, $V_{{\mathcal{L}}}(x_{0})=x_{0}^{\top}K^{\star}x_{0}$, where $K^{\star}:=\gamma({\mathcal{N}}({\mathcal{L}}))$ and ${\mathcal{N}}({\mathcal{L}}):=\langle{\mathcal{L}}\cap\textup{Ker}(K^{-})\;|\;A(K^{-})\rangle\cap{\mathcal{X}}^{+}(A(K^{-}))$. 3. (iii)

For all $x_{0}\in{\mathbb{R}}^{n}$, there exists an optimal input $u^{\star}$ if and only if $\textup{Ker}(\Delta)\subset{\mathcal{L}}\cap\textup{Ker}(K^{-})$. 4. (iv)

If $\textup{Ker}(\Delta)\subset{\mathcal{L}}\cap\textup{Ker}(K^{-})$, then for each $x_{0}\in{\mathbb{R}}^{n}$, there exists exactly one optimal input $u^{\star}$, and it is given by the feedback $u^{\star}=-B^{\top}K^{\star}x$.

This paper can be regarded as a generalization of the previous result to the stabilizable case. That is, we require weaker assumptions for the sufficient condition of well-posedness to be able to provide the form of the value function, necessary and sufficient conditions for attainability, and the form of the optimal control. Our new assumptions involve the stabilizability of $(A,B)$ rather than controllability, and the existence of a negative semidefinite solution to the ARI rather than imposing that specifically $K^{-}$, a solution to the ARE, is negative semidefinite on ${\mathcal{L}}$. Because necessary and sufficient conditions for well-posedness are still an open problem, note that we have not attempted to generalize our second condition in terms of the existence of an ARI solution that is negative semidefinite on ${\mathcal{L}}$. Regardless, the main technical obstacle is that there is no direct generalization of Theorem 3.3 to the stabilizable case; indeed the minimal solution $K^{-}$ may not exist in this case.

Now supposing that $(A,B)$ is stabilizable, we can write the system (1) in the Kalman controllability decomposition. Let $\mathcal{C}=\langle A\;|\;\textup{Im}(B)\rangle\subset{\mathbb{R}}^{n}$ be the controllable subspace with dimension $n_{1}\leq n$. Also, let ${\mathcal{X}}_{2}$ be any complement such that

[TABLE]

Then the system matrices have the block form:

[TABLE]

It can be shown that coordinate transformations only affect the solutions $K\in\partial\Gamma$ of the $\text{(LQCP)}_{\mathcal{L}}~{}$(in any endpoint case) up to a congruent transformation, so there is no loss of generality to assume that $(A,B)$ already has the form (13). If we write the symmetric matrices $Q$ and $K$ in block form

[TABLE]

then $\phi(K)$ also can be decomposed in block form:

[TABLE]

We note that $\phi(K)$ is symmetric, and $\phi_{1}(K_{1})$ is defined below in (17). Let

[TABLE]

Then $\phi(K)=0$ gives rise to three equations

[TABLE]

The first equation (17) is a quadratic equation with $(A_{1},B_{1})$ controllable. Its solutions $K_{1}$ are decoupled from $K_{12}$ and $K_{2}$, so this lower order ($n_{1}\times n_{1}$) ARE equation can be solved first. The relevant solution sets are denoted as:

[TABLE]

Using any solution $K_{1}\in\partial\Gamma_{1}$, if it exists, (18) is a linear (Sylvester) equation for $K_{12}$ which may have no solutions, infinitely many solutions, or a unique solution. The third equation (19) is also a linear (Sylvester) equation. Using any solution $K_{12}$, if it exists, gives a unique solution to $K_{2}$. To see this, recall that if $M_{1}\in{\mathbb{R}}^{n_{1}\times n_{1}}$, $M_{2}\in{\mathbb{R}}^{n_{2}\times n_{2}}$, and $M_{3}\in{\mathbb{R}}^{n_{1}\times n_{2}}$ are given matrices, then the Sylvester equation $M_{1}X+XM_{2}=M_{3}$ has a unique solution $X\in{\mathbb{R}}^{n_{1}\times n_{2}}$ exactly when $\sigma(M_{1})\cap\sigma(-M_{2})=\emptyset$ [5]. Because stabilizability of $(A,B)$ implies $\sigma(A_{2})\subset{\mathbb{C}}^{-}$, then by applying the Sylvester solvability criteria to (19), we have that $\sigma(A_{2}^{\top})\cap\sigma(-A_{2})=\emptyset$, and so $K_{2}$ is unique for any given $K_{12}$.

In preparation for characterizing the existence and form of the value function analogously to Theorem 3.4 (i) and (ii), we consider existence of extremal solutions in $\partial\Gamma$. It is known that when $(A,B)$ is stabilizable, then the maximal solution $K^{+}\in\partial\Gamma$ exists, whereas the minimal solution $K^{-}$ may not exist.

Theorem 3.5** (Theorem 2.1, [10]; Theorem 7.9.3, p. 195, [11]).**

*Suppose $(A,B)$ is stabilizable and $\partial\Gamma\neq\emptyset$. Then the unique maximal solution $K^{+}\in\partial\Gamma$ exists. Moreover, $\sigma(A(K^{+}))\subset{\mathbb{C}}^{-}\cup{\mathbb{C}}^{0}$. *

To obtain a generalization of Theorem 3.4 to the stabilizable case, one of the major steps in the sequel is to apply Theorem 3.4 to the controllable subsystem $(A_{1},B_{1})$ and its ARE (17). Theorem 3.4 requires that the minimal solution $K_{1}^{-}$ of (17) exists and is negative semidefinite on ${\mathcal{L}}$ within the controllable subspace. The following lemma provides for the existence of this minimal, negative semidefinite solution.

Lemma 3.6**.**

*Suppose $(A,B)$ is stabilizable, $\Gamma_{-}\neq\emptyset$, and the state space is decomposed as in (12). Then the minimal solution $K_{1}^{-}\in\partial\Gamma_{1-}$ exists. *

Proof 3.7**.**

*Let $K\in\Gamma_{-}$ so that $\phi(K)\geq 0$ and $K\leq 0$. Consider $K$, $Q$, and $\phi(K)$ in block form (14)-(15). Applying Theorem .1 to both $K$ and $\phi(K)$, we obtain $\phi_{1}(K_{1})\geq 0$ and $K_{1}\leq 0$, which implies $K_{1}\in\Gamma_{1-}\neq\emptyset$. Since also $(A_{1},B_{1})$ is controllable, we can apply Theorem 3.1(i) to conclude $K_{1}^{+},K_{1}^{-}\in\Gamma_{1}$, the maximal and minimal solutions, exist. Moreover $\partial\Gamma_{1}\neq\emptyset$ and its maximal and minimal elements are precisely $K_{1}^{+}$ and $K_{1}^{-}$. Because $K_{1}\leq 0$, $K_{1}^{-}\in\Gamma_{1}$ is minimal, and $K_{1},K^{-}_{1}\in\Gamma_{1}$, we have that $K_{1}^{-}\leq K_{1}\leq 0$. That is, $K^{-}_{1}\in\partial\Gamma_{1-}$, as desired. *

4 Solution of the $\text{(LQCP)}_{\mathcal{L}}~{}$

In this section we present the solution of the $\text{(LQCP)}_{\mathcal{L}}~{}$. That is, we give sufficient conditions for well-posedness, the form of the value function, necessary and sufficient conditions for attainability, and form of the optimal control. We assume that ${\mathcal{L}}\subset{\mathbb{R}}^{n}$ is a given subspace. Well-posedness and the form of the value function are addressed through the following sufficient condition, which are also found in [7, 8].

Assumption 2**.**

*We assume that $(A,B)$ is stabilizable and $\Gamma_{-}\neq\emptyset$. *

The following theorem states that the value function is given in terms of a quadratic form of a particular solution to the ARE.

Theorem 4.1** (Theorem 2.1 [7], Lemma 5 [12]).**

*Consider the $\text{(LQCP)}_{\mathcal{L}}~{}$and suppose Assumption 2 holds. Then there exists a unique $K^{\star}\in\partial\Gamma$ such that for all $x_{0}\in{\mathbb{R}}^{n}$, $V_{{\mathcal{L}}}(x_{0})=x_{0}^{\top}K^{\star}x_{0}$. *

Next we turn to the form of $K^{\star}$. Our approach is to choose a suitable basis based on the Kalman controllability decomposition (12) and on Theorem 3.3, following the same method in [17]. Then we systematically determine each of the blocks of $K^{\star}$. First we determine $K^{\star}_{1}$ using results from [16]; second, we compute $K_{12}^{\star}$ assuming $K^{\star}_{1}$ is known; finally, we compute $K_{2}^{\star}$ assuming $K_{12}^{\star}$ is known. Now we give a more detailed roadmap on how the technical results are obtained.

The choice of $K^{\star}_{1}$ is resolved by applying Theorem 3.4 to the controllable subsystem. We construct a smaller optimal control problem on the controllable subsystem. Intuitively, the smaller optimal control problem should be equivalent to the original $\text{(LQCP)}_{\mathcal{L}}~{}$for initial conditions in the controllable subspace. After proving this equivalence, we apply Theorem 3.4 to obtain $K^{\star}_{1}=\overline{K}_{1}$, where $\overline{K}_{1}$ is defined in (22) below. Next, we fix the choice of $K^{\star}_{1}$ that solves (17) and turn to the solution set of (18). Generally, this linear Sylvester equation may have an infinite number of solutions, making the choice of $K_{12}^{\star}$ nontrivial to determine. However, once $K_{12}^{\star}$ is determined, then $K_{2}^{\star}$ is uniquely determined from the linear Sylvester equation (19), since $(A,B)$ is stabilizable. Thus $K_{12}^{\star}$ is the main obstacle. Interestingly, under a restrictive regularity assumption introduced in [10], the solution set of (18) collapses to a single element. On the other hand, Theorem 4.1 states that $K_{12}^{\star}$ exists without the regularity assumption. We forego the assumption and search for a more general principle that can resolve the choice of $K_{12}^{\star}$.

Our approach involves exploiting the structure within the Kalman controllability decomposition, similarly as in [17]. Based on a modal decomposition of $A_{1}(\overline{K}_{1})$, the Sylvester equation (18) with $K_{1}=K^{\star}_{1}$ splits into three decoupled linear Sylvester equations (34)-(36). The problematic part of $K_{12}^{\star}$, denoted $K_{12,1}^{*}$ is then isolated to (34) only. Regarding the solution of (34), it is well known (see Theorem 10.13 of [18]) that for stabilizable systems with positive semidefinite cost in the free endpoint case, the solution of the ARE is given by the smallest positive semidefinite solution in $\partial\Gamma$. Also, $0\in\Gamma$ if and only if $Q\geq 0$ (see for example equation (1.16) of [8]) and so $0\in\Gamma_{-}$ and $x_{0}^{\top}0x_{0}=0$ gives a lower bound on the value function. Using the previous two observations, we find through repeated trials that $K_{12}^{\star}=0$ in the positive semidefinite case. At this point we make a guess that the same form of $K_{12}^{\star}$ would arise in the indefinite case. Finally, we unambiguously deduce that $K_{12}^{\star}=0$.

Once we have fully characterized the form of $K^{\star}$, obtaining necessary and sufficient conditions for attainability follows analogously to the proof presented in [17, 16]. We require only a few augmentations to account for the uncontrollable (but stable) dynamics. Now we proceed to the actual development.

The first step is to fix a suitable basis so that the blocks of $K^{\star}$ can be computed. Consider the Kalman controllability decomposition (12), and suppose Assumption 2 holds. Then by Lemma 3.6, the unique minimal solution $K^{-}_{1}\in\partial\Gamma_{1}\neq\emptyset$ exists and $K^{-}_{1}\leq 0$. Similarly, because $(A_{1},B_{1})$ is controllable and $\partial\Gamma_{1}\neq\emptyset$, we can apply Theorem 3.1 to obtain the unique maximal solution $K^{+}_{1}\in\partial\Gamma_{1}$. Let $\Delta_{1}:=K_{1}^{+}-K_{1}^{-}$ be the gap associated with (17), the ARE in the controllable subspace. Following [17, 16], we can further decompose the controllable subspace based on Theorem 3.3. To that end, define the following subspaces of ${\mathbb{R}}^{n_{1}}$:

[TABLE]

Here and for the remainder of this section, for simplicity we do not notationally differentiate a subspace that can belong to various vector spaces of different dimensions. For example, although technically ${\mathcal{L}}\cap{\mathcal{C}}\subset{\mathbb{R}}^{n}$, we can view ${\mathcal{L}}_{1}$ as a subspace of ${\mathbb{R}}^{n_{1}}\sim{\mathcal{C}}$.

Let $P_{{\mathcal{N}}_{1}({\mathcal{L}}_{1})}:{\mathbb{R}}^{n_{1}}\rightarrow{\mathcal{N}}_{1}({\mathcal{L}}_{1})$ be the projection onto ${\mathcal{N}}_{1}({\mathcal{L}}_{1})$ along $\Delta_{1}^{-1}({\mathcal{N}}_{1}({\mathcal{L}}_{1})^{\perp})$. Because ${\mathcal{N}}_{1}({\mathcal{L}}_{1})$ is an $A_{1}(K_{1}^{-})$-invariant subspace contained in ${\mathcal{X}}^{+}(A_{1}(K_{1}^{-}))$ for any ${\mathcal{L}}_{1}$, we can apply Theorem 3.3 to obtain a solution $\overline{K}_{1}\in\partial\Gamma_{1}$ of the ARE of the form

[TABLE]

Following Theorem 3.3, define the following subspaces in ${\mathcal{C}}\sim{\mathbb{R}}^{n_{1}}$:

[TABLE]

Then the state space decomposition (12) splits further into

[TABLE]

Let $n_{1,i}:=\text{dim}({\mathcal{X}}_{1,i})$ for $i=1,2,3$ so that $n_{1}=n_{1,1}+n_{1,2}+n_{1,3}\leq n$. Without loss of generality (after a change of coordinates), the system matrices have the block form

[TABLE]

The cost matrix $Q$ and each $K\in\Gamma$ have the block form

[TABLE]

Our goal is to compute all of the blocks in (28) for $K=K^{\star}$. First we resolve the choice of $K^{\star}_{1}$.

Theorem 4.2**.**

*Consider the $\text{(LQCP)}_{\mathcal{L}}~{}$and suppose Assumption 2 holds. Then in the state space decomposition (12), $K^{\star}_{1}=\overline{K}_{1}$, as given in (22). *

Proof 4.3**.**

Since $(A,B)$ is stabilizable, without loss of generality, $(A,B)$ has the form (13), and $Q$ and $K$ have the block form (14). Defining $x:=(x_{1},x_{2})$, the Kalman controllability decomposition is

[TABLE]

The controllable subspace is ${\mathcal{C}}=\{x\in{\mathbb{R}}^{n}\;|\;x_{2}=0\}$. If $x_{2,0}=0$, then for all $t\geq 0$, $x_{2}(t)=0$ and $x(t)\in{\mathcal{C}}$. Thus, we can define a new $\text{(LQCP)}_{\mathcal{L}_{1}}~{}$on ${\mathcal{C}}$ with dynamics $\dot{x}_{1}=A_{1}x_{1}+B_{1}u$, $x_{1}(0)=x_{1,0}$, and $(A_{1},B_{1})$ is controllable. The cost function is $J_{1T}(x_{1,0},u):=\int_{0}^{T}\omega_{1}(x_{1}(t;x_{1,0},u),u(t))\,dt$ with $\omega_{1}(x_{1},u):=x_{1}^{\top}Q_{1}x_{1}+u^{\top}u$. Let ${\mathcal{L}}_{1}={\mathcal{L}}\cap{\mathcal{C}}$ be the terminal subspace and let $d_{1{\mathcal{L}}_{1}}:{\mathbb{R}}^{n_{1}}\rightarrow[0,\infty)$ be the distance function. The input spaces are

[TABLE]

The optimal cost is $V_{1{\mathcal{L}}_{1}}(x_{1,0}):=\inf\{\lim_{T\rightarrow\infty}J_{1T}(x_{1,0},u)\;|\;u\in{\mathcal{U}}_{1{\mathcal{L}}_{1}}(x_{1,0})\}$. The ARE for the $\text{(LQCP)}_{\mathcal{L}_{1}}~{}$is $\phi_{1}(K_{1})=0$ as in (17) with solution set $\partial\Gamma_{1}$. Consider any initial condition $x_{0}=(x_{1,0},0)\in{\mathcal{C}}$ and any control $u\in L_{2,loc}^{m}({\mathbb{R}}^{+})$. Then $x(t;x_{0},u)=(x_{1}(t;x_{1,0},u),0)$ and $\omega(x(t;x_{0},u),u(t))=\omega_{1}(x_{1}(t;x_{1,0},u),u(t))$, so for all $T\geq 0$, $J_{T}(x_{0},u)=J_{1T}(x_{1,0},u)$. Consequently, we have ${\mathcal{U}}(x_{0})={\mathcal{U}}_{1}(x_{1,0})$. Also, $\lim_{t\rightarrow\infty}d_{{\mathcal{L}}}(x(t;x_{0},u))=0$ is equivalent to $\lim_{t\rightarrow\infty}d_{1{\mathcal{L}}_{1}}(x_{1}(t;x_{1,0},u))=0$. Thus ${\mathcal{U}}_{{\mathcal{L}}}(x_{0})={\mathcal{U}}_{1{\mathcal{L}}_{1}}(x_{1,0})$. With all the above, we conclude that $V_{{\mathcal{L}}}(x_{0})=V_{1{\mathcal{L}}_{1}}(x_{1,0})$ for $x_{0}=(x_{1,0},0)\in{\mathcal{C}}$.

*Since $(A_{1},B_{1})$ is controllable, we can apply the results of [16] to solve the $\text{(LQCP)}_{\mathcal{L}_{1}}~{}$. Since $\Gamma_{-}\neq\emptyset$, we can apply Lemma 3.6 to get that the minimal solution $K_{1}^{-}\in\partial\Gamma_{1-}$ exists. Since $K_{1}^{-}\leq 0$, from (11) it follows that $K_{1}^{-}$ is negative semidefinite on ${\mathcal{L}}_{1}$. By Theorem 3.4(ii), $V_{1{\mathcal{L}}_{1}}(x_{1,0})=x_{1,0}^{\top}\overline{K}_{1}x_{1,0}$ with $\overline{K}_{1}$ given in (22). Since we have already shown that $V_{{\mathcal{L}}}(x_{0})=V_{1{\mathcal{L}}_{1}}(x_{1,0})$ for $x_{0}=(x_{1,0},0)\in{\mathcal{C}}$, it can be easily shown that $K^{\star}_{1}=\overline{K}_{1}$. *

To resolve the remaining blocks of $K^{\star}$, we recall some results from [17]. For this to apply, we continue to assume that the state space is decomposed according to (26). It was shown in (5.5) and (5.7) of [17] that $\overline{K}_{1}$ in (22) and the closed-loop system matrix $A_{1}(\overline{K}_{1})$ using $\overline{K}_{1}$ have the form

[TABLE]

where $\sigma(\overline{A}_{1,11})\subset{\mathbb{C}}^{+}$, $\sigma(\overline{A}_{1,22})\subset{\mathbb{C}}^{0}$, and $\sigma(\overline{A}_{1,33})\subset{\mathbb{C}}^{-}$. For the choice of $K_{1}=\overline{K}_{1}$ and substituting (27), (28), and (33), the second ARE equation (18) splits into three linear Sylvester equations:

[TABLE]

Using these facts, we can now resolve the remaining blocks of $K^{\star}$. The main difficulty is that (34) may have an infinite number of solutions for the $K_{12,1}$ block since $\sigma(\overline{A}_{1,11}^{\top})\cap\sigma(-A_{2})$ is not necessarily empty. The key insight is that $K^{\star}_{12,1}$ can be unambiguously determined by invoking Theorem 4.6(ii) given below, that any negative semidefinite solution $K_{N}\in\Gamma_{-}$ to the ARI provides a lower bound to the value function. In order to utilize this property to resolve the choice of $K_{12,1}^{\star}$, the next lemma describes the block structure of any $K_{N}\in\Gamma_{-}$.

Lemma 4.4**.**

Suppose Assumption 2 holds and the state space is decomposed as in (26). Then for all $K_{N}\in\Gamma_{-}$, $K_{N}$ has the block form

[TABLE]

Proof 4.5**.**

Let $K_{N}\in\Gamma_{-}$ have the block form in (28). Since $\Gamma_{-}\neq\emptyset$ and $(A,B)$ is stabilizable, we can apply Lemma 3.6 to obtain that the minimal solution $K_{1}^{-}\in\partial\Gamma_{1-}$ exists. Also $\partial\Gamma_{1-}\subset\partial\Gamma_{1}\neq\emptyset$. Because $K_{N}\in\Gamma_{-}\subset\Gamma$, by Theorem .1 we establish that its upper left block satisfies $K_{1}\in\Gamma_{1}$. Since $(A_{1},B_{1})$ is controllable and $\partial\Gamma_{1}\neq\emptyset$, we can apply Theorem 3.1(i) to get that the maximal solution $K_{1}^{+}\in\partial\Gamma_{1}$ also exists. Moreover, Theorem 3.1(i) also implies that $K_{1}^{-},K_{1}^{+}\in\Gamma_{1}$, and consequently $K_{1}^{-}\leq K_{1}\leq K_{1}^{+}$. Since $\partial\Gamma_{1}\neq\emptyset$, it has been shown (see equation (5.6) in [17] and equation (5.4) in [16]) that $K_{1}^{+}$, $K_{1}^{-}$, and $\Delta_{1}$ have the block form

[TABLE]

where $K_{1,22}^{+}=K_{1,22}^{-}$, $K_{1,23}^{+}=K_{1,23}^{-}$, and $\Delta_{1,33}=K_{1,33}^{+}-K_{1,33}^{-}$. Now consider $K_{1}\geq K_{1}^{-}$ in block form, assuming the decomposition of $K_{1}^{-}$ in (37). We have

[TABLE]

Using Theorem .1, we find $K_{1,11}\geq 0$. Since $K_{N}\in\Gamma_{-}$ by assumption, $K_{N}\leq 0$. Applying Theorem .1 to $K_{N}=\begin{bmatrix}K_{1,11}&*\\ *&*\end{bmatrix}$, we get $K_{1,11}\leq 0$. Thus $K_{1,11}=0$. Now consider again $K_{1}^{-}\leq K_{1}\leq K_{1}^{+}$ with the information that $K_{1,11}=0$:

[TABLE]

where we have $K_{1,22}^{+}=K_{1,22}^{-}$ and $K_{1,23}^{+}=K_{1,23}^{-}$ as in (37). We claim that $K_{1,12}=0$, $K_{1,13}=0$, $K_{1,22}=K_{1,22}^{-}$, and $K_{1,23}=K_{1,23}^{-}$. First, we have

[TABLE]

Applying Theorem .1 again, we get $(I-00^{\dagger})\begin{bmatrix}K_{1,12}&K_{1,13}\end{bmatrix}=0$, so that $K_{1,12}=0$ and $K_{1,13}=0$. Then $K_{1}-K_{1}^{-}\geq 0$ reduces to

[TABLE]

which implies by Theorem .1 that

[TABLE]

Similarly, $K_{1}^{+}-K_{1}\geq 0$ gives

[TABLE]

Applying Theorem .1 to the previous two statements, we get $K_{1,22}^{-}\leq K_{1,22}\leq K_{1,22}^{-}$, so $K_{1,22}=K_{1,22}^{-}$. Then rewriting the previous inequality (38)

[TABLE]

Applying Theorem .1, we get $(I-00^{\dagger})(K_{1,23}^{-}-K_{1,23})=0$, so $K_{1,23}=K_{1,23}^{-}$. So far we have for $K_{N}\in\Gamma_{-}$

[TABLE]

Then $-K_{N}\geq 0$ has the block form:

[TABLE]

*Applying Lemma .2, we get $K_{12,1}=0$. *

In the next result we completely characterize the form of $K^{\star}$. Before proceeding with this result, we collect some well known results about the cost function.

Theorem 4.6**.**

Consider the system (1) with the cost function (2) - (3). Let $x_{0}\in{\mathbb{R}}^{n}$, $T\geq 0$, and $u\in L_{2,loc}^{m}({\mathbb{R}}^{+})$.

1. (i)

Let $K\in\partial\Gamma$. Then $J_{T}(x_{0},u)=\int_{0}^{T}\|u(t)+B^{\top}Kx(t)\|^{2}\,dt+x_{0}^{\top}Kx_{0}-x^{\top}(T)Kx(T)$, where $x(t):=x(t;x_{0},u)$. 2. (ii)

For all $x_{0}\in{\mathbb{R}}^{n}$ and $K_{N}\in\Gamma_{-}$, $V_{{\mathcal{L}}}(x_{0})\geq x_{0}^{\top}K_{N}x_{0}$. 3. (iii)

Suppose Assumption 2 holds. If $J(x_{0},u)=x_{0}^{\top}K^{\star}x_{0}$, then $u=-B^{\top}K^{\star}x$ and $\lim_{T\rightarrow\infty}x^{\top}(T)K^{\star}x(T)=0$.

Proof 4.7**.**

*Statement (i) is standard. See for instance [19] or [17]. Statement (ii) is Proposition 1.8 of [7]. See also Lemma 4.4 of [17]. Statement (iii) is Theorem 2.8(c) of [7]. See also the proof of Theorem 5.1(iii) in [17]. *

Theorem 4.8**.**

Consider the $\text{(LQCP)}_{\mathcal{L}}~{}$and suppose Assumption 2 holds. Then in the state space decomposition (26), $K^{\star}\in\partial\Gamma$ has the form

[TABLE]

*where $K_{12,2}^{\star}$ is the unique solution to (35), $K_{12,3}^{\star}$ is the unique solution to (36), and $K_{2}^{\star}$ is the unique solution to (19) with $K_{12}=K_{12}^{\star}$. *

Proof 4.9**.**

By Theorem 4.2, $K^{\star}_{1}=\overline{K}_{1}$ with the form of $\overline{K}_{1}$ given in (22). By Theorem 4.1, $K^{\star}\in\partial\Gamma$. Next we consider (18). Using the decompositions above and with the choice $K_{1}=\overline{K}_{1}$, the second ARE equation (18) splits into (34), (35), and (36). Since $\sigma(\overline{A}_{1,22})\subset{\mathbb{C}}^{0}$, $\sigma(\overline{A}_{1,33})\subset{\mathbb{C}}^{-}$, and $\sigma(-A_{2})\subset{\mathbb{C}}^{+}$, (35) and (36) have unique solutions $K_{12,2}^{\star}$ and $K_{12,3}^{\star}$, respectively [5]. Similarly, (19) has a unique solution $K_{2}^{\star}$, assuming $K_{12}=K_{12}^{\star}$. At this point we know that $K^{\star}$ has the block form:

[TABLE]

Comparing to (39), it remains only to show that $K_{12,1}^{\star}=0$. By Theorem 4.1, $V_{{\mathcal{L}}}(x_{0})=x_{0}^{\top}K^{\star}x_{0}$. Let $K_{N}\in\Gamma_{-}$. By Theorem 4.6(ii), for all $x_{0}\in{\mathbb{R}}^{n}$, $V_{{\mathcal{L}}}(x_{0})=x_{0}^{\top}K^{\star}x_{0}\geq x_{0}^{\top}K_{N}x_{0}$; that is, $K^{\star}\geq K_{N}$. Using the block form of $K^{\star}$ in (40) and the block form of $K_{N}$ in Lemma 4.4, we have

[TABLE]

*Applying Lemma .2 yields that $K_{12,1}^{\star}=0$, as desired. *

Remark 4.10**.**

We observe from the form of $K^{\star}$ that $K^{\star}_{12,1}=0$. If we substitute $K^{\star}_{12,1}=0$ into (34), we get that $Q_{12,1}=0$. One can derive the fact that $Q_{12,1}=0$ via a separate argument, and this provides an independent validation of our result that $K^{\star}_{12,1}=0$. Suppose Assumption 2 holds. Take any symmetric $K$ with the special form:

[TABLE]

We decompose $A$ and $B$ as in (27). Using a result analogous to equation (5.2) in [17], it can be shown that ${\mathcal{N}}_{1}({\mathcal{L}}_{1})$ is $A_{1}$-invariant, and this implies $A_{1,21}=A_{1,31}=0$. Then by direct computation $\phi(K)$ has the form:

[TABLE]

Now choose the upper left block of the above $K$ to be $K_{1}^{-}\in\partial\Gamma$. By (37) this choice is consistent with the form of $K$ above. Since the upper left block of $\phi(K)$ is written as $\phi_{1}(K_{1})$ and we know that $\phi_{1}(K_{1}^{-})=0$, it immediately follows that $Q_{1,11}=0$, $Q_{1,12}=0$, and $Q_{1,13}=0$. Next, since $\Gamma_{-}\neq\emptyset$, let $K_{N}\in\Gamma_{-}$. By Lemma 4.4, $K_{N}$ has the special form above. Then we have

[TABLE]

*By applying Lemma .2, we conclude that $Q_{12,1}=0$. *

We conclude this section by applying Theorem 4.8 to obtain necessary and sufficient conditions for attainability of the $\text{(LQCP)}_{\mathcal{L}}~{}$. Remarkably, the attainability result depends only on the controllable subspace.

Theorem 4.11**.**

*Suppose Assumption 2 holds and the state space is decomposed as in (12). Then the $\text{(LQCP)}_{\mathcal{L}}~{}$is attainable if and only if $\textup{Ker}(\Delta_{1})\subset{\mathcal{L}}_{1}\cap\textup{Ker}(K_{1}^{-})$. *

Proof 4.12**.**

Due to Assumption 2, we may further assume that the state space is decomposed according to (26). Let $W_{1}\in{\mathbb{R}}^{n_{1}\times n_{1}}$ be a matrix such that $\textup{Ker}(W_{1})={\mathcal{L}}_{1}$ and let $d_{1{\mathcal{L}}_{1}}:{\mathbb{R}}^{n_{1}}\rightarrow[0,\infty)$ be the distance function in ${\mathbb{R}}^{n_{1}}$ to ${\mathcal{L}}_{1}$. Since ${\mathcal{X}}_{1,1}=\langle{\mathcal{L}}_{1}\cap\textup{Ker}(K_{1}^{-})\;|\;A_{1}(K_{1}^{-})\rangle\cap{\mathcal{X}}^{+}(A_{1}(K_{1}^{-}))$, we have ${\mathcal{X}}_{1,1}\subset\langle\textup{Ker}(W_{1})\cap\textup{Ker}(K_{1}^{-})\;|\;A_{1}(K_{1}^{-})\rangle\subset\textup{Ker}(W_{1})\cap\textup{Ker}(K_{1}^{-})=\textup{Ker}\left(\begin{bmatrix}K_{1}^{-}\\ W_{1}\end{bmatrix}\right)$. We claim

[TABLE]

Proof of Claim: Let $x_{1}\in{\mathcal{X}}_{1,1}$. Then $x_{1}\in\textup{Ker}\left(\begin{bmatrix}K_{1}^{-}\\ W_{1}\end{bmatrix}\right)=:\textup{Ker}\left(\begin{bmatrix}D_{1}&D_{2}&D_{3}\end{bmatrix}\right)$. Also since $x_{1}\in{\mathcal{X}}_{1,1}$, in coordinates it has the form $x_{1}=(x_{1,1},0,0)$. Then $\begin{bmatrix}D_{1}&D_{2}&D_{3}\end{bmatrix}x_{1}=D_{1}x_{1,1}=0$. Since $x_{1,1}$ is arbitrary, we get $D_{1}=0$, as desired.

$(\Rightarrow)$*  Suppose the $\text{(LQCP)}_{\mathcal{L}}~{}$is attainable. Let $x_{0}\in{\mathbb{R}}^{n}$. By definition there exists $u^{\star}\in{\mathcal{U}}_{{\mathcal{L}}}(x_{0})$ such that $V_{{\mathcal{L}}}(x_{0})=J(x_{0},u^{\star})$. By Theorem 4.1 we know $V_{{\mathcal{L}}}(x_{0})=x_{0}^{\top}K^{\star}x_{0}$ where $K^{\star}\in\partial\Gamma$, and by Theorem 4.8, $K^{\star}$ is given in (39). Now we can apply Theorem 4.6(iii) to get $u^{\star}=-B^{\top}K^{\star}x$. The closed-loop dynamics are $\dot{x}=A(K^{\star})x$. Let $x:=(x_{1},x_{2}):=(x_{1,1},x_{1,2},x_{1,3},x_{2})$ according to the decomposition (26). Then using the block form of $A_{1}(K^{\star}_{1})=A(\overline{K}_{1})$ in (33), we have*

[TABLE]

where $\sigma(\overline{A}_{1,11})\subset{\mathbb{C}}^{+}$, $\sigma(\overline{A}_{1,22})\subset{\mathbb{C}}^{0}$, $\sigma(\overline{A}_{1,33})\subset{\mathbb{C}}^{-}$, and by stabilizability, $\sigma(A_{2})\subset{\mathbb{C}}^{-}$. Using the variation of constants formula we get that at $t=T$

[TABLE]

Since $\sigma(A_{2})\subset{\mathbb{C}}^{-}$, $\lim_{T\rightarrow\infty}x_{2}(T)=0$. Using (43) for $i=3$, $\sigma(\overline{A}_{1,33})\subset{\mathbb{C}}^{-}$, and the fact that $\lim_{T\rightarrow\infty}x_{2}(T)=0$, we also get $\lim_{T\rightarrow\infty}x_{1,3}(T)=0$. Now using (39), the block form of $K_{1}^{-}$ given in (37), and the fact that $K_{1,33}^{+}=\Delta_{1,33}+K_{1,33}^{-}$, we have

[TABLE]

Using this expression combined with the fact that $\lim_{T\rightarrow\infty}x_{1,3}(T)=0$, $\lim_{T\rightarrow\infty}x_{2}(T)=0$, and $\lim_{T\rightarrow\infty}x^{\top}(T)K^{\star}x(T)=0$ from Theorem 4.6(iii), we get

[TABLE]

Now we observe that $\lim_{T\rightarrow\infty}2x_{1,2}^{\top}(T)K_{12,2}^{\star}x_{2}(T)=0$ because $\sigma(\overline{A}_{1,22})\subset{\mathbb{C}}^{0}$ and $\sigma(A_{2})\subset{\mathbb{C}}^{-}$. Returning to (45), this implies that also $\lim_{T\rightarrow\infty}x_{1}^{\top}(T)K_{1}^{-}x_{1}(T)=0$.

We have assumed that $u^{\star}\in{\mathcal{U}}_{{\mathcal{L}}}(x_{0})$ and $(A,B)$ is stabilizable. Therefore, $\lim_{T\rightarrow\infty}d_{{\mathcal{L}}\cap{\mathcal{C}}}(x(T))=0$, and thus within the controllable subspace $\lim_{T\rightarrow\infty}d_{1{\mathcal{L}}_{1}}(x_{1}(T))=0$. Since ${\mathcal{L}}_{1}=\textup{Ker}(W_{1})$, $\lim_{T\rightarrow\infty}W_{1}x_{1}(T)=0$. Meanwhile by Lemma 3.6, $K_{1}^{-}\leq 0$. Since $\lim_{T\rightarrow\infty}x_{1}^{\top}(T)K_{1}^{-}x_{1}(T)=0$, by taking the limit in (11) we have that $\lim_{T\rightarrow\infty}K_{1}^{-}x_{1}(T)=0$. Overall, we have $\lim_{T\rightarrow\infty}\begin{bmatrix}K_{1}^{-}\\ W_{1}\end{bmatrix}x_{1}(T)=0$. Using (41), this gives $\lim_{T\rightarrow\infty}\left(D_{2}x_{1,2}(T)+D_{3}x_{1,3}(T)\right)=0$. We already know that $\lim_{T\rightarrow\infty}x_{1,3}(T)=0$, so we get $\lim_{T\rightarrow\infty}D_{2}x_{1,2}(T)=0$. However, $\sigma(\overline{A}_{1,22})\subset{\mathbb{C}}^{0}$ and $x_{1,2}(0)$ is arbitrary, so $D_{2}=0$. Finally, we observe that if $x_{1}\in{\mathcal{X}}_{1,2}$, then $\begin{bmatrix}0&0&D_{3}\end{bmatrix}x_{1}=0$ since $x_{1}=(0,x_{1,2},0)$. That is, ${\mathcal{X}}_{1,2}\subset\textup{Ker}\left(\begin{bmatrix}0&0&D_{3}\end{bmatrix}\right)$. In sum, we have

[TABLE]

$(\Leftarrow)$*  Suppose that $\textup{Ker}(\Delta_{1})\subset{\mathcal{L}}_{1}\cap\textup{Ker}(K_{1}^{-})$. Let $x_{0}\in{\mathbb{R}}^{n}$. To show attainability, we must find an optimal control. Consider the candidate $u^{c}:=-B^{\top}K^{\star}x$, where $K^{\star}$ is given in (39). We must show $V_{{\mathcal{L}}}(x_{0})=J(x_{0},u^{c})$ and $u^{c}\in{\mathcal{U}}_{{\mathcal{L}}}(x_{0})$. The closed-loop dynamics using $u^{c}$ are given in (42). Following the same arguments as above we have that $\lim_{T\rightarrow\infty}x_{2}(T)=0$ and $\lim_{T\rightarrow\infty}x_{1,3}(T)=0$. By assumption, $\textup{Ker}(\Delta_{1})\subset{\mathcal{L}}_{1}\cap\textup{Ker}(K_{1}^{-})$. From above, ${\mathcal{L}}_{1}\cap\textup{Ker}(K_{1}^{-})=\textup{Ker}\left(\begin{bmatrix}K_{1}^{-}\\ W_{1}\end{bmatrix}\right)=\textup{Ker}\left(\begin{bmatrix}0&D_{2}&D_{3}\end{bmatrix}\right)$. We claim that $D_{2}=0$. To see this, let $x_{1}\in\textup{Ker}(\Delta_{1})={\mathcal{X}}_{1,2}$. Then $x_{1}=(0,x_{1,2},0)$. Since $\textup{Ker}(\Delta_{1})\subset\textup{Ker}\left(\begin{bmatrix}0&D_{2}&D_{3}\end{bmatrix}\right)$ we have $\begin{bmatrix}0&D_{2}&D_{3}\end{bmatrix}x_{1}=D_{2}x_{1,2}=0$. Since $x_{1,2}$ is arbitrary, $D_{2}=0$. Using the block form of $K_{1}^{-}$ in (37), we have*

[TABLE]

This implies $K_{1,22}^{-}=K_{1,23}^{-}=0$. Now we observe $K^{\star}\in\partial\Gamma$ by Theorem 4.1 and $u^{c}\in L_{2,loc}^{m}({\mathbb{R}}^{+})$ for any fixed $T\geq 0$. Therefore, we can apply Theorem 4.6(i) with $K=K^{\star}$ and $u=u^{c}$ to get

[TABLE]

We claim that $\lim_{T\rightarrow\infty}x^{\top}(T)K^{\star}x(T)=0$. Using the expansion of $x(T)^{\top}K^{\star}x(T)$ given in (44), and the fact that $\lim_{T\rightarrow\infty}x_{2}(T)=0$ and $\lim_{T\rightarrow\infty}x_{1,3}(T)=0$, we get $\lim_{T\rightarrow\infty}x^{\top}(T)K^{\star}x(T)=\lim_{T\rightarrow\infty}x_{1}^{\top}(T)K_{1}^{-}x_{1}(T)$. Using the available information about the block form of $K_{1}^{-}$ and that $\lim_{T\rightarrow\infty}x_{1,3}(T)=0$, we find

[TABLE]

Returning to (47), we have $\lim_{T\rightarrow\infty}J_{T}(x_{0},u^{c})=J(x_{0},u^{c})=x_{0}^{\top}K^{\star}x_{0}$, as desired.

Finally, we must show $u^{c}\in{\mathcal{U}}_{{\mathcal{L}}}(x_{0})$, and particularly $\lim_{T\rightarrow\infty}d_{{\mathcal{L}}}(x(T))=0$. Since $\lim_{T\rightarrow\infty}x_{1,3}(T)=0$, we have that

[TABLE]

*Thus, $\lim_{T\rightarrow\infty}W_{1}x_{1}(T)=0$, which implies $\lim_{T\rightarrow\infty}d_{1{\mathcal{L}}_{1}}(x_{1}(T))=0$. Since ${\mathcal{L}}_{1}={\mathcal{L}}\cap{\mathcal{C}}$ and $\lim_{T\rightarrow\infty}x_{2}(T)=0$, we have $\lim_{T\rightarrow\infty}d_{{\mathcal{L}}}(x(T))=0$. Thus, $u^{c}\in{\mathcal{U}}_{{\mathcal{L}}}(x_{0})$, as desired. *

We collect all of the previous results to obtain the culminating result on the solution of the $\text{(LQCP)}_{\mathcal{L}}~{}$. It is a generalization of Theorem 3.4 for the case of $(A,B)$ controllable to the case when $(A,B)$ is stabilizable.

Theorem 4.13**.**

Consider the $\text{(LQCP)}_{\mathcal{L}}~{}$. Suppose Assumption 2 holds and the state space is decomposed as in (12). Then we have

1. (i)

The problem is well-posed. 2. (ii)

For all $x_{0}\in{\mathbb{R}}^{n}$, $V_{{\mathcal{L}}}(x_{0})=x_{0}^{\top}K^{\star}x_{0}$. 3. (iii)

For all $x_{0}\in{\mathbb{R}}^{n}$, the problem is attainable if and only if $\textup{Ker}(\Delta_{1})\subset{\mathcal{L}}_{1}\cap\textup{Ker}(K_{1}^{-})$. 4. (iv)

If the problem is attainable, then for each $x_{0}\in{\mathbb{R}}^{n}$, there exists exactly one optimal input $u^{\star}$, and it is given by $u^{\star}=-B^{\top}K^{\star}x$.

Proof 4.14**.**

*Statements (i) and (ii) follow from Theorem 4.1. The form of $K^{\star}$ follows from Theorem 4.8. Statement (iii) is an immediate consequence of Theorem 4.11. Statement (iv) follows from Theorem 4.6 (iii). *

5 Discussion

In this section we discuss several special cases of our main result. This includes a comparison with classical results in the positive semidefinite case. First, we consider the special case when ${\mathcal{N}}_{1}({\mathcal{L}}_{1})=0$ which was also treated in Theorem 6.1 of [17]. From our experience it is only in exceptional cases that ${\mathcal{N}}_{1}({\mathcal{L}}_{1})\neq 0$. The following result shows that when ${\mathcal{N}}_{1}({\mathcal{L}}_{1})=0$, then $K^{\star}=K^{+}$, the maximal solution in $\partial\Gamma$. This result has practical significance because there are many powerful algorithms for numerically finding the maximal solution of the ARE.

Theorem 5.1**.**

*Consider the $\text{(LQCP)}_{\mathcal{L}}~{}$, suppose that Assumption 2 holds, and that the state space decomposed as in (12). Then ${\mathcal{N}}_{1}({\mathcal{L}}_{1})=0$ if and only if $K^{*}=K^{+}$, where $K^{+}\in\partial\Gamma$ is the maximal solution. *

Proof 5.2**.**

(Only if)  Suppose ${\mathcal{N}}_{1}({\mathcal{L}}_{1})=0$. By Theorem 4.8, $K^{\star}:=\begin{bmatrix}K^{\star}_{1}&K^{\star}_{12}\\ K^{\star\top}_{12}&K^{\star}_{2}\end{bmatrix}\in\partial\Gamma$, where $K_{1}^{\star}=\overline{K}_{1}=\gamma({\mathcal{N}}_{1}({\mathcal{L}}_{1}))$. By assumption, $P_{{\mathcal{N}}_{1}({\mathcal{L}}_{1})}=0$, and then (22) gives $K_{1}^{\star}=K_{1}^{+}$, where $K_{1}^{+}$ is the maximal solution in $\partial\Gamma_{1}$. By Theorem 3.1(ii), we also know $K_{1}^{+}\in\partial\Gamma_{1}$ is the unique maximal solution such that $\sigma(A_{1}(K_{1}^{+}))\subset{\mathbb{C}}^{-}\cup{\mathbb{C}}^{0}$. Furthermore, by stabilizability, $\sigma(A_{2})\subset{\mathbb{C}}^{-}$. Therefore, $\sigma(A_{1}(K_{1}^{+}))\cap\sigma(-A_{2})=\emptyset$, so $K_{12}^{\star}$ is the unique solution of the Sylvester equation (18). Similarly, since $\sigma(A_{2}^{\top})\cap\sigma(-A_{2})=\emptyset$, $K_{2}^{\star}$ is the unique solution of the Sylvester equation (19).

Meanwhile, since $\partial\Gamma\neq\emptyset$, by Theorem 3.5, the maximal solution $K^{+}\in\partial\Gamma$ exists and satisfies $\sigma(A(K^{+}))\subset{\mathbb{C}}^{-}\cup{\mathbb{C}}^{0}$. We claim $K^{\star}=K^{+}$. Let $K^{+}=\begin{bmatrix}K_{1}&K_{12}\\ K^{\top}_{12}&K_{2}\end{bmatrix}$ in block form. Since $K^{+}\in\partial\Gamma$, we have that $K_{1}\in\partial\Gamma_{1}$. Using (13), $\sigma(A(K^{+}))=\sigma(A_{1}(K_{1}))\uplus\sigma(A_{2})\subset{\mathbb{C}}^{-}\cup{\mathbb{C}}^{0}$. Then since $\sigma(A_{2})\subset{\mathbb{C}}^{-}$, we have $\sigma(A_{1}(K_{1}))\subset{\mathbb{C}}^{-}\cup{\mathbb{C}}^{0}$. However, by Theorem 3.1(ii), $K_{1}\in\partial\Gamma_{1}$ and $\sigma(A_{1}(K_{1}))\subset{\mathbb{C}}^{-}\cup{\mathbb{C}}^{0}$ together imply $K_{1}=K_{1}^{+}=K_{1}^{\star}$, the unique maximal solution in $\partial\Gamma_{1}$. It immediately follows that $K_{12}=K^{\star}_{12}$ and $K_{2}=K^{\star}_{2}$, as desired.

*(If)  Suppose $K^{*}=K^{+}$, the maximal solution in $\partial\Gamma$. By writing $K^{+}$ in block form, $K^{+}=\begin{bmatrix}K_{1}&K_{12}\\ K_{12}^{\top}&K_{2}\end{bmatrix}$, we have $K_{1}=K^{\star}_{1}$. We also have that $K_{1}=K_{1}^{+}$ is the maximal solution in $\partial\Gamma_{1}$ using an argument analogous to the one above. That is, using (13), $\sigma(A(K^{+}))=\sigma(A_{1}(K_{1}))\uplus\sigma(A_{2})$. By Theorem 3.5, $\sigma(A(K^{+}))\subset{\mathbb{C}}^{-}\cup{\mathbb{C}}^{0}$. Since $\sigma(A_{2})\subset{\mathbb{C}}^{-}$, we get $\sigma(A_{1}(K_{1}))\subset{\mathbb{C}}^{-}\cup{\mathbb{C}}^{0}$. Then by Theorem 3.1(ii), $K_{1}=K_{1}^{+}\in\partial\Gamma_{1}$. Meanwhile by Theorem 4.2, $\overline{K}_{1}=K^{\star}_{1}$. Putting this altogether, we have that $\overline{K}_{1}=K^{\star}_{1}=K_{1}^{+}$. Finally, using $\overline{K}_{1}=K_{1}^{+}$ in (22) gives that $P_{{\mathcal{N}}_{1}({\mathcal{L}}_{1})}=0$, so ${\mathcal{N}}_{1}({\mathcal{L}}_{1})=0$. *

Next we discuss how Theorem 4.13 recovers well known results for the free-endpoint and fixed-endpoint problems when $Q$ is positive semidefinite and $(A,B)$ is stabilizable. First, we observe that when $Q\geq 0$, then $\phi(0)\geq 0$ so $0\in\Gamma_{-}\neq\emptyset$. Therefore, Assumption 2 holds. We also assume that the state space is decomposed as in (26) wherever needed.

The main results on the free endpoint problem are summarized in Theorem 10.13 in [18]. In particular, when ${\mathcal{L}}={\mathbb{R}}^{n}$, $V_{{\mathcal{L}}}(x_{0})=x_{0}^{\top}P^{-}x_{0}$, where $P^{-}\geq 0$ is the smallest positive semidefinite solution to the ARE, and the optimal control is $u^{\star}(t)=-B^{\top}P^{-}x(t)$. We would like to verify that our Theorem 4.13 recovers these results. We will show that when $Q\geq 0$, $K^{\star}$ given in (39) satisfies $K^{\star}=P^{-}$. To aid in this endeavor, we invoke a result from [17]. Let $\partial\Gamma_{1+}:=\{K_{1}\in\partial\Gamma_{1}~{}|~{}K_{1}\geq 0\}$.

Theorem 5.3** (Theorem 6.3 [17]).**

*Assume $(A_{1},B_{1})$ is controllable and $\partial\Gamma_{1-}\neq\emptyset$. Then the following hold: if $\partial\Gamma_{1+}\neq\emptyset$, then (i) $\overline{K}_{1}\in\partial\Gamma_{1+}$ and (ii) $K_{1}\in\partial\Gamma_{1+}$ implies $\overline{K}_{1}\leq K_{1}$. *

Lemma 5.4**.**

*Consider the $\text{(LQCP)}_{\mathcal{L}}~{}$. Suppose $(A,B)$ is stabilizable, ${\mathcal{L}}={\mathbb{R}}^{n}$, and $Q\geq 0$. Then $K^{\star}=P^{-}$. *

Proof 5.5**.**

We begin by applying Theorem 5.3 to show that $\overline{K}_{1}$ is the smallest solution in $\partial\Gamma_{1+}$. To that end, we must show that $\partial\Gamma_{1-}\neq\emptyset$ and $\partial\Gamma_{1+}\neq\emptyset$. First, since Assumption 2 holds, we can apply Lemma 3.6 to get $K_{1}^{-}\in\partial\Gamma_{1-}$ exists, so $\partial\Gamma_{1-}\neq\emptyset$. Second, because $Q\geq 0$, we know $\phi(0)\geq 0$, so $0\in\Gamma_{-}$. By Theorem 4.1, $V_{{\mathcal{L}}}(x)=x^{\top}K^{\star}x$. Applying Theorem 4.6(ii) with $K_{N}=0$, we get $x^{\top}K^{\star}x\geq x^{\top}0x=0$, for all $x\in{\mathbb{R}}^{n}$, so $K^{\star}\geq 0$. That is, $K^{\star}\in\partial\Gamma_{+}$. By Theorem .1, this implies $K_{1}^{\star}\geq 0$, so $K_{1}^{\star}=\overline{K}_{1}\in\partial\Gamma_{1+}\neq 0$. Now we can apply Theorem 5.3 to get $K_{1}^{\star}=\overline{K}_{1}$ is the smallest solution in $\partial\Gamma_{1+}$.

It remains to show that $K^{\star}=P^{-}$ is the smallest solution in $\partial\Gamma_{+}$. To arrive at a contradiction, suppose there exists $K\in\partial\Gamma_{+}$ such that $K\neq K^{\star}$ and $K\leq K^{\star}$. There are two cases. First, suppose $K\in\partial\Gamma_{+}$ with $K\leq K^{\star}$ such that $K_{1}\neq K^{\star}_{1}$, where $K_{1}$ is the upper left block of $K$. Since $K\in\partial\Gamma$, $\phi(K)=0$, so $\phi_{1}(K_{1})=0$, implying $K_{1}\in\partial\Gamma_{1}$. By Theorem .1, $K\geq 0$ implies $K_{1}\geq 0$, so $K_{1}\in\partial\Gamma_{1+}$. Again by Theorem .1, $K\leq K^{\star}$ implies $K_{1}\leq K^{\star}_{1}$. Thus, we have $K_{1}\in\partial\Gamma_{1+}$ such that $K_{1}\leq K^{\star}_{1}$, which contradicts that $K^{\star}_{1}$ is the smallest solution in $\partial\Gamma_{1+}$.

For the second case, suppose $K\in\partial\Gamma_{+}$ with $K\leq K^{\star}$ such that $K_{1}=K^{\star}_{1}$. By (33), $K$ has the form

[TABLE]

*Since $K\geq 0$, we can apply Lemma .2 to find that $K_{12,1}=0$. Then since $K_{1}=K^{\star}_{1}$, $K_{12,1}=K_{12,1}^{\star}=0$, and $\phi(K)=0$, the solutions for $K_{12,2}$ and $K_{12,3}$ are unique and match $K_{12,2}^{\star}$ and $K_{12,3}^{\star}$, respectively. Thus $K=K^{\star}$, a contradiction. We conclude that $K^{\star}$ is the smallest solution in $\partial\Gamma_{+}$. This proves that for the free endpoint case when $Q\geq 0$ that $V_{{\mathcal{L}}}(x_{0})=x_{0}^{\top}P^{-}x_{0}$. Also, Theorem 4.13 (iv) gives the optimal control $u(t)=-B^{\top}P^{-}x(t)$ since $P^{-}=K^{\star}$. *

Next we consider attainability in the free endpoint case. Since Assumption 2 holds, we can apply Theorem 4.13(iii). In the free endpoint problem, ${\mathcal{L}}_{1}={\mathcal{C}}$, so by Theorem 4.13(iii), the problem is attainable if and only if $\textup{Ker}(\Delta_{1})\subset\textup{Ker}(K_{1}^{-})$. By Proposition 6.4 of [17], the latter condition always holds. Thus, we recover the well-known fact that for the free endpoint case in the positive semidefinite case, the problem is always attainable.

Now we discuss the fixed endpoint problem. The main results are summarized in Theorem 10.18 in [18]. In particular, when ${\mathcal{L}}=0$, $V_{{\mathcal{L}}}(x_{0})=x_{0}^{\top}P^{+}x_{0}$, where $P^{+}\geq 0$ is the largest positive semidefinite solution to the ARE, and the optimal control is $u^{\star}(t)=-B^{\top}P^{+}x(t)$. We would like to verify that our Theorem 4.13 recovers these results. We must show that when $Q\geq 0$, then $K^{\star}=P^{+}$. For the fixed endpoint problem, ${\mathcal{L}}_{1}=0$, so ${\mathcal{N}}_{1}({\mathcal{L}}_{1})=0$. The desired result is then immediately obtained from Theorem 5.1.

Now we consider attainability in the fixed endpoint case. The well-known necessary and sufficient conditions for attainability in the positive semidefinite case, stated in Theorem 10.18(iii) of [18], is that every eigenvalue of $A$ on the imaginary axis is $(Q,A)$ observable. We must show that this statement is equivalent to our attainability result in Theorem 4.13(iii), which for the fixed-endpoint case requires that $\textup{Ker}(\Delta_{1})\subset 0\cap\textup{Ker}(K_{1}^{-})$, or equivalently, $\Delta_{1}>0$. This connection is resolved by the following result, whose proof is found in the Appendix.

Theorem 5.6**.**

*Suppose $(A,B)$ is stabilizable and $Q\geq 0$. Then every eigenvalue of $A$ on the imaginary axis is $(Q,A)$ observable if and only if $\Delta_{1}>0$. *

The final verification of our result in the fixed endpoint case is to show that the closed-loop system, $\dot{x}(t)=(A-BB^{\top}K^{+})x(t)=A(K^{+})x(t)$, is asymptotically stable, thereby recovering Theorem 10.18(v) in [18]. Note that $A(K)=A-BB^{\top}K=\begin{bmatrix}A_{1}(K_{1})&*\\ 0&A_{2}\end{bmatrix}$ so that $\sigma(A(K))=\sigma(A_{1}(K_{1}))\uplus\sigma(A_{2})$. By Theorem 5 in [19], we have that $\Delta_{1}>0$ if and only if $\sigma(A_{1}(K_{1}^{+}))\subset{\mathbb{C}}^{-}$. Since $\sigma(A_{2})\subset{\mathbb{C}}^{-}$ by stabilizability and $\Delta_{1}>0$ by attainability, we have $\sigma(A(K^{+}))\subset{\mathbb{C}}^{-}$, as desired.

6 Conclusion

In this paper we address a problem in the area of linear quadratic optimal control which has been open for the last 20 years. Specifically, we consider the regular, infinite-horizon, stability-modulo-a-subspace, indefinite LQ problem when the dynamics are stabilizable. Previous works have also addressed this problem, but under the restrictive assumption that the dynamics are controllable. The generalization from controllable to stabilizable dynamics is significant in that there is a lack of structure in the solutions of the algebraic Riccati equation in the stabilizable case. Consequently the connection between the ARE solution set and the LQ problem under consideration has remained elusive. We resolved this gap by combining a suitable sufficient condition for a finite optimal cost with a specific decomposition to unambiguously deduce the correct form of the optimal cost and control. The determination of necessary and sufficient conditions for a finite value function in the regular, infinite-horizon, stability-modulo-a-subspace, indefinite LQ problem is still open. As future work, we are also interested in applying our result to reachability problems, namely by employing an indefinite cost functional on a stabilizable linear system to characterize the convergence of trajectories to a nontrivial subspace over the infinite time horizon.