Positive semidefinite interval of matrix pencil and its applications for the generalized trust region subproblems

TL;DR

This paper characterizes the positive semidefinite interval of matrix pencils and applies these findings to simplify solving the hard case of generalized trust region subproblems.

Contribution

It provides a detailed analysis of the positive semidefinite interval of matrix pencils, especially when matrices are not simultaneously diagonalizable, and applies this to improve GTRS solutions.

Findings

The interval $I_{\succeq}(A,B)$ can be empty, a singleton, or an interval.

When both matrices are singular, the interval can be decomposed into block diagonals.

The approach simplifies solving the hard case of GTRS by reducing it to linear systems or smaller GTRS problems.

Abstract

We are concerned with finding the set $I_{\succeq}(A,B)$ of real values $\mu$ such that the matrix pencil $A+\mu B$ is positive semidefinite. If $A, B$ are not simultaneously diagonalizable via congruence (SDC), $I_{\succeq}(A,B)$ either is empty or has only one value $\mu.$ When $A, B$ are SDC, $I_{\succeq}(A,B),$ if not empty, can be a singleton or an interval. Especially, if $I_{\succeq}(A,B)$ is an interval and at least one of the matrices is nonsingular then its interior is the positive definite interval $I_{\succ}(A,B).$ If $A, B$ are both singular, then even $I_{\succeq}(A,B)$ is an interval, its interior may not be $I_{\succ}(A,B),$ but $A, B$ are then decomposed to block diagonals of submatrices $A_1, B_1$ with $B_1$ nonsingular such that $I_{\succeq}(A,B)=I_{\succeq}(A_1,B_1).$ Applying $I_{\succeq}(A,B),$ the hard-case of the generalized trust-region subproblem (GTRS) can be dealt with by only solving a system of linear equations or reduced to the easy-case of a GTRS of smaller size.

Equations

$$I_{⪰}(A,B)=\{\mu \in \mathbb{R}:A+\mu B⪰0\}$$

$$\hskip 17.07182pt\begin{array}[]{lll}\lambda^{*}=&\min&f(x)=x^{T}Ax+2a^{T}x\\ &{\rm s.t.}&g(x)=x^{T}Bx+2b^{T}x+c\leq 0,\end{array}$$

$$\nabla f(x^{\ast })+{\mu }^{\ast }\nabla g(x^{\ast })=0,$$

$${\mu }^{\ast }g(x^{\ast })=0;$$

$$P^T(A+\mu B)P=\text{diag}(({\lambda }_1+\mu )B_1,({\lambda }_2+\mu )B_2\dots ,({\lambda }_k+\mu )B_k),$$

$$U^{T}BU=\text{diag}(B_1,0)\text{ and }{U}^{T}AU=\text{diag}(A_1,0)$$

$$U^{T}BU=\text{diag}(B_1,0)\text{ and }{U}^{T}AU=\text{diag}(A_1,A_4),$$

$$J:=P^{-1}{B}^{-1}AP=\text{diag}({\lambda }_1{I}_{m_1},\dots ,{\lambda }_k{I}_{m_k}),$$

$$K=\text{diag}(C({\lambda }_1),C({\lambda }_2),\cdots ,C({\lambda }_k)),$$

$$K_{i_{j}}(\lambda_{i})=\left(\begin{array}[]{cccccc}\lambda_{i}&1&0&\cdots&\cdots&0\\ 0&\lambda_{i}&1&\cdots&\cdots&0\\ \cdots&\cdots&\cdots&\cdots&\cdots&\cdots\\ \cdots&\cdots&\cdots&\cdots&\cdots&\cdots\\ 0&0&\cdots&\cdots&\lambda_{i}&1\\ 0&0&\cdots&\cdots&0&\lambda_{i}\end{array}\right)_{(ij)},\hskip 28.45274ptj=1,2,\cdots,t_{i}.$$

$$\dim S_i=\dim C({\lambda }_i).$$

$$P^{T}BP=\text{diag}(B_1,B_2\dots ,B_k),$$

$$P^{T}AP=P^{T}BP.J=\text{diag}({\lambda }_1{B}_1,{\lambda }_2{B}_2\dots ,{\lambda }_k{B}_k).$$

$$P^T(A+\mu B)P=\text{diag}(({\lambda }_1+\mu )B_1,({\lambda }_2+\mu )B_2\dots ,({\lambda }_k+\mu )B_k).$$

$$\begin{array}{r}({\lambda }_i+\mu )B_i⪰0,i=1,2,\dots ,k.\end{array}$$

$$\{\begin{array}{l}{\lambda }_i+\mu \ge 0,\forall i=1,2,\dots ,t,\\ {\lambda }_i+\mu \le 0,\forall i=t+1,\dots ,k.\end{array}$$

$$\{\begin{array}{l}{\lambda }_i+\mu \ge 0,\forall i=1,2,\dots ,t-1\\ {\lambda }_t+\mu =0\\ {\lambda }_i+\mu \le 0,\forall i=t+1,\dots ,k.\end{array}$$

$$\{\begin{array}{l}{\lambda }_i+\mu \le 0\\ {\lambda }_j+\mu \ge 0\end{array}$$

$$\{\begin{array}{l}{\lambda }_i+\mu =0\\ {\lambda }_j+\mu \ge 0\end{array}$$

$$U^{T}BU=\text{diag}(B_1,0),$$

$$U^{T}AU=\text{diag}(A_1,A_3),$$

$$\hat{B}=Q_1^{T}B{Q}_1=\text{diag}(B_1,0)$$

$$\hat{A}=Q_1^{T}A{Q}_1=\left(\begin{array}{cc}{M}_1& M_2\\ M_2^T& M_3\end{array}\right)$$

$$H^T\hat{B}H=\left(\begin{array}{cc}{H}_1^T{B}_1{H}_1& H_1^T{B}_1{H}_2\\ H_2^T{B}_1{H}_1& H_2^T{B}_1{H}_2\end{array}\right)=\left(\begin{array}{cc}{H}_1^T{B}_1{H}_1& 0\\ 0& 0\end{array}\right),$$

$$H^T\hat{A}H=\left(\begin{array}{cc}{H}_1^T{M}_1{H}_1+H_3^T{M}_2^T{H}_1+H_1^T{M}_2{H}_3& H_1^T{M}_2{H}_4\\ H_4^T{M}_2^T{H}_1& 0\end{array}\right)$$

$$H^T\hat{A}H=\left(\begin{array}{cc}{H}_1^T{M}_1{H}_1+H_3^T{M}_2^T{H}_1+H_1^T{M}_2{H}_3& 0\\ 0& 0\end{array}\right).$$

$$\tilde{A}:=U_1^T\hat{A}{U}_1=\left(\begin{array}{cc}{M}_1& M_{2}P\\ (M_{2}P{)}^T& P^T{M}_{3}P\end{array}\right)=\left(\begin{array}{ccc}{M}_1& A_4& A_5\\ A_4^T& A_3& 0\\ A_5^T& 0& 0\end{array}\right),$$

$$U_2=\left(\begin{array}{ccc}{I}_p& 0& 0\\ -A_3^{-1}{A}_4^T& I_r& 0\\ 0& 0& I_{q-r}\end{array}\right)\text{ and }U=Q_1{U}_1{U}_2.$$

$$U^{T}BU=U_2^T{U}_1^T(Q_1^{T}B{Q}_1))U_1{U}_2=\hat{B},$$

$$U^{T}AU=U_2^T\tilde{A}{U}_2=\left(\begin{array}{ccc}{M}_1-A_4{A}_3^{-1}{A}_4^T& 0& A_5\\ 0& A_3& 0\\ A_5^T& 0& 0\end{array}\right).$$

$$U^{T}BU=\text{diag}(B_1,0),U^{T}AU=\left(\begin{array}{ccc}{A}_1& 0& A_5\\ 0& A_3& 0\\ A_5^T& 0& 0\end{array}\right).$$

Full text

url]nguyenthingan@ttn.edu.vn

1 Introduction

Let $\mathcal{S}^{n}$ be the space of $n\times n$ real symmetric matrices and $A,B\in\mathcal{S}^{n}.$ In this paper we compute the set

[TABLE]

and then apply it for solving the following optimization problem

[TABLE]

where $a,b\in\mathbb{R}^{n}$ are $n-$ dimension vectors, $c\in\mathbb{R}.$ This problem is referred as the generalized trust region subproblem (GTRS) since it contains the trust region subproblem (TRS), when $B=I$ is the identity matrix, $b=0$ and $c=-1,$ as a special case.

Our study is inspired by the following results obtained by Moré [10]: i) a vector $x^{*}\in\mathbb{R}^{n}$ is a global optimizer of the GTRS (1) if and only if $g(x^{*})\leq 0$ and there exists $\mu^{*}\in I_{\succeq}(A,B)\cap[0,\infty)$ such that

[TABLE]

and ii) if the set defined by $I_{\succ}(A,B)=\{\mu\in\mathbb{R}:A+\mu B\succ 0\}$ is nonempty then it is an open interval and the function $\varphi(\mu):=g[x(\mu)]$ is strictly decreasing on $I_{\succ}(A,B),$ unless $x(\mu)$ uniquely solved from (2) is constant on $I_{\succ}(A,B).$ These two results suggest that $\mu^{*}$ can be found efficiently whenever $I_{\succeq}(A,B)$ is computed. Unfortunately, in literature, results on $I_{\succeq}(A,B)$ are very limited. The earliest study on computing $I_{\succeq}(A,B)$ was, perhaps, by Caron and Gould [3], where the authors suppose that $A$ is positive semidefinite and $B$ is of rank one or two. Song [11] computed $I_{\succeq}(A,B)$ under the assumption that $\mathcal{R}(B)\subset\mathcal{R}(A),$ where $\mathcal{R}(\cdot)$ denotes the range of a matrix. Moré [10], Adachi et al. [1] computed $I_{\succeq}(A,B)$ under the assumption that $I_{\succ}(A,B)\neq\emptyset.$ Out of above conditions, computing $I_{\succeq}(A,B)$ is still an open question so far. However, if $I_{\succ}(A,B)\neq\emptyset,$ $I_{\succeq}(A,B)$ is then the closure of $I_{\succ}(A,B)$ and computing $I_{\succeq}(A,B)$ is much easier since the matrices $A,B$ are then simultaneously diagonalizable via congruence (SDC), i.e., there exists a nonsingular matrix $P$ such that $P^{T}AP$ and $P^{T}BP$ are all diagonal, please see [9]. Moré [10] proposed an algorithm for finding $\mu^{*}$ under the assumption that $I_{\succ}(A,B)$ is nonempty, while the case $I_{\succ}(A,B)=\emptyset$ is referred as the hard-case. The assumption $I_{\succ}(A,B)\neq\emptyset$ is so strictive that it restricts the GTRS (1) into a very special class which always attains a unique optimal solution. Moreover, if $I_{\succ}(A,B)\neq\emptyset$ then $A,B$ are SDC and the GTRS (1) is equivalently reformulated as a convex second-order cone programming (SOCP) [2]. This result is then extended that any bounded GTRS can be reformulated as an SOCP even the SDC condition fails [5]. The assumption $I_{\succ}(A,B)\neq\emptyset$ also allows to solve GTRS (1) by solving only one eigenpair of a generalized eigenvalue problem of $(2n+1)\times(2n+1)$ dimension [1]. A more recent result by Jiang and Li can solve GTRS (1) even in linear time $O(n)$ but under a stricter condition that there exist $\mu\in(0,1]$ and $\lambda_{\rm min}(B)\leq-\xi<0$ such that $\mu A+(1-\mu)B\succeq\xi I,$ here $\lambda_{\rm min}(B)$ is the smallest eigenvalue of $B$ [6]. Hsia et al. [8] solve the GTRS (1) when $A,B$ are SDC, while the case when $A,B$ are not SDC is still referred as the hard-case.

In this paper, we show that not only when $I_{\succ}(A,B)\neq\emptyset$ but also even $I_{\succ}(A,B)=\emptyset$ the interval $I_{\succeq}(A,B)$ can be computed by only solving a generalized eigenvalue problem of $n\times n$ dimension. Specifically, we show that if $A,B$ are not SDC then $I_{\succeq}(A,B)$ either is empty or has only one point: $I_{\succeq}(A,B)=\{\mu\}.$ Such a value $\mu$ can be found efficiently and then checked whether $\mu^{*}=\mu.$ If $A,B$ are SDC and $B$ is nonsingular, then the matrix pencil $A+\mu B$ can always be decomposed into the form

[TABLE]

where $B_{i}$ are $m_{i}\times m_{i}$ symmetric matrices, $\lambda_{1}>\lambda_{2}>\ldots>\lambda_{k}$ are distinct eigenvalues of the matrix $B^{-1}A.$ The set $I_{\succeq}(A,B)$ is then computed quickly since $A+\mu B\succeq 0$ is equivalent to $(\lambda_{i}+\mu)B_{i}\succeq 0$ for all $i=1,2,\ldots,k.$ If $B$ is singular and $A$ is nonsingular, we can decompose $B,A$ to the taking the forms $U^{T}BU=\texttt{diag}(B_{1},0),$ $U^{T}AU=\texttt{diag}(A_{1},A_{3}),$ where $B_{1},A_{1}$ are symmetric of the same size, $B_{1}$ is nonsingular such that if $A_{3}\succ 0$ then $I_{\succeq}(A,B)=I_{\succeq}(A_{1},B_{1}),$ otherwise $I_{\succeq}(A,B)=\emptyset.$ Especially, if $I_{\succeq}(A,B)$ has more than one point, then $I_{\succ}(A,B)\neq\emptyset$ and $I_{\succ}(A,B)={\rm int}(I_{\succeq}(A,B)),$ please see Corollary 1 below. If $A,B$ are SDC and both are singular, then there always exists a nonsingular matrix $U$ such that $A,B$ are decomposed to either the following form

[TABLE]

or

[TABLE]

where $A_{4}$ is diagonal and, in both cases, $A_{1},B_{1}$ are diagonal and of the same size, $B_{1}$ is nonsingular. If $A,B$ are decomposed to (5) and $A_{4}$ has even one diagonal negative element then $I_{\succeq}(A,B)=\emptyset.$ Otherwise, in both cases, $I_{\succeq}(A,B)=I_{\succeq}(A_{1},B_{1})$ with $B_{1}$ nonsingular. To apply $I_{\succeq}(A,B)$ for solving GTRS (1), we consider the set of candidates of Lagrange multipliers $I=I_{\succeq}(A,B)\cap[0,\infty).$

If $I=\emptyset,$ (1) has no optimal solution since it is unbounded from below; 2. 2.

If $I$ is singleton: $I=\{\mu\},$ we need only to solve linear equations to check whether $\mu^{*}=\mu;$ 3. 3.

If $I$ is an interval and $I_{\succ}(A,B)\neq\emptyset,$ then there is a unique optimal Lagrange multiplier $\mu^{*}\in I.$ If $\mu^{*}$ is not an endpoint of $I,$ a bisection algorithm can find $\mu^{*}$ in the interior of $I$ since the function $\varphi(\mu)$ is strictly decreasing on $I.$ 4. 4.

If $I$ is an interval and $I_{\succ}(A,B)=\emptyset,$ $A,B$ are converted to the form either (4) or (5) and $I_{\succ}(A_{1},B_{1})\neq\emptyset.$ In this case, the GTRS (1) is either unbounded below or reduced to a GTRS of $p$ variables with matrices $A_{1},B_{1}$ such that $I_{\succ}(A_{1},B_{1})\neq\emptyset.$

2 Computing the positive semidefinite interval $I_{\succeq}(A,B)$

In this section, we show computing $I_{\succeq}(A,B)$ in two separate cases: $A,B$ are SDC and $A,B$ are not SDC. For the former case, we need first the following result.

Lemma 1** ([7]).**

Let $A,B\in\mathcal{S}^{n}$ and $B$ be nonsingular. Then $A,B$ are SDC if and only if there is a nonsingular real matrix $P$ such that $P^{-1}B^{-1}AP$ is a real diagonal matrix.

Now, if $A,B$ are SDC and $B$ is nonsingular, by Lemma 1, there is a nonsingular matrix $P$ such that

[TABLE]

is a diagonal matrix, where $\lambda_{1},\lambda_{2},\ldots,\lambda_{k}$ are the $k$ distinct eigenvalues of $B^{-1}A,$ $I_{m_{t}}$ is the identity matrix of size $m_{t}\times m_{t}$ and $m_{1}+m_{2}+\ldots+m_{k}=n.$ We can suppose without loss of generality that $\lambda_{1}>\lambda_{2}>\ldots>\lambda_{k}.$ Using $J$ together with the following result we show how to simultaneously decompose $A$ and $B$ into block diagonals.

Lemma 2** ([13]).**

Let $K$ be a Jordan matrix of form

[TABLE]

where $C(\lambda_{i})=\texttt{diag}(K_{i_{1}}(\lambda_{i}),K_{i_{2}}(\lambda_{i}),\cdots,K_{i_{t_{i}}}(\lambda_{i})),i=1,2,\ldots,k,$ are Jordan blocks associated with eigenvalue $\lambda_{i}$ and

[TABLE]

For a symmetric matrix $S,$ if $SK$ is symmetric, then $S$ is block diagonal and $S=\texttt{diag}(S_{1},S_{2},\cdots,S_{k})$ with

[TABLE]

Observe that $P^{T}BP.J=P^{T}AP$ and $P^{T}AP$ is symmetric. Lemma 2 indicates that $P^{T}BP$ is a block diagonal matrix with the same partition as $J.$ That is

[TABLE]

where $B_{t}$ is real symmetric matrices of size $m_{t}\times m_{t}$ for every $t=1,2,\ldots,k.$ We now have

[TABLE]

Both (6) and (7) show that $A,B$ are now decomposed into the same block structure and the matrix pencil $A+\mu B$ now becomes

[TABLE]

The requirement $A+\mu B\succeq 0$ is then equivalent to

[TABLE]

Using (9) we compute $I_{\succeq}(A,B)$ as follows.

Theorem 1**.**

Suppose $A,B\in\mathcal{S}^{n}$ are SDC and $B$ is nonsingular.

If $B\succ 0$ then $I_{\succeq}(A,B)=[-\lambda_{k},+\infty);$ 2. 2.

If $B\prec 0$ then $I_{\succeq}(A,B)=(-\infty,-\lambda_{1}];$ 3. 3.

If $B$ is indefinite then

• (i)

if $B_{1},B_{2},\ldots,B_{t}\succ 0$ and $B_{t+1},B_{t+2},\ldots,B_{k}\prec 0$ for some $t\in\{1,2,\ldots,k\},$ then $I_{\succeq}(A,B)=[-\lambda_{t},-\lambda_{t+1}].$

• (ii)

if $B_{1},B_{2},\ldots,B_{t-1}\succ 0,$ $B_{t}$ is indefinite and $B_{t+1},B_{t+2},\ldots,B_{k}\prec 0,$ then $I_{\succeq}(A,B)=\{-\lambda_{t}\},$

• (iii)

in other cases, that is either $B_{i},B_{j}$ are indefinite for some $i\neq j$ or $B_{i}\prec 0,B_{j}\succ 0$ for some $i<j$ or $B_{i}$ is indefinite and $B_{j}\succ 0$ for some $i<j,$ then $I_{\succeq}(A,B)=\emptyset.$

Proof.

If $B\succ 0$ then $B_{i}\succ 0~{}\forall i=1,2,\ldots,k.$ The inequality (9) is then equivalent to $\lambda_{i}+\mu\geq 0~{}\forall i=1,2,\ldots,k.$ Since $\lambda_{1}>\lambda_{2}>\ldots>\lambda_{k},$ we need only $\mu\geq-\lambda_{k}.$ This shows $I_{\succeq}(A,B)=[-\lambda_{k},+\infty).$ 2. 2.

Similarly, if $B\prec 0$ then $B_{i}\prec 0~{}\forall i=1,2,\ldots,k.$ The inequality (9) is then equivalent to $\lambda_{i}+\mu\leq 0~{}\forall i=1,2,\ldots,k.$ Then $I_{\succeq}(A,B)=(-\infty,-\lambda_{1}].$ 3. 3.

The case $B$ is indefinite:

• (i)

if $B_{1},B_{2},\ldots,B_{t}\succ 0$ and $B_{t+1},B_{t+2},\ldots,B_{k}\prec 0$ for some $t\in\{1,2,\ldots,k\},$ the inequality (9) then implies

[TABLE]

Since $\lambda_{1}>\lambda_{2}>\ldots>\lambda_{k},$ we have $I_{\succeq}(A,B)=[-\lambda_{t},-\lambda_{t+1}].$

• (ii)

if $B_{1},B_{2},\ldots,B_{t-1}\succ 0,$ $B_{t}$ is indefinite and $B_{t+1},B_{t+2},\ldots,B_{k}\prec 0$ for some $t\in\{1,2,\ldots,k\}.$ The inequality (9) then implies

[TABLE]

Since $\lambda_{1}>\lambda_{2}>\ldots>\lambda_{k},$ we have $I_{\succeq}(A,B)=\{-\lambda_{t}\}.$

• (iii)

if $B_{i},B_{j}$ are indefinite, (9) implies $\lambda_{i}+\mu=0$ and $\lambda_{j}+\mu=0.$ This cannot happen since $\lambda_{i}\neq\lambda_{j}.$ If $B_{i}\prec 0$ and $B_{j}\succ 0$ for some $i<j,$ then

[TABLE]

implying $-\lambda_{j}\leq\mu\leq-\lambda_{i}.$ This also cannot happen since $\lambda_{i}>\lambda_{j}.$ Finally, if $B_{i}$ is indefinite and $B_{j}\succ 0$ for some $i<j.$ Again, by (9),

[TABLE]

implying $\lambda_{i}\leq\lambda_{j}.$ This also cannot happen. So $I_{\succeq}(A,B)=\emptyset$ in these all three cases.

∎

The proof of Theorem 1 indicates that if $A,B$ are SDC, $B$ is nonsingular and $I_{\succeq}(A,B)$ is an interval then $I_{\succ}(A,B)$ is nonempty. In that case we have $I_{\succ}(A,B)={\rm int}(I_{\succeq}(A,B)),$ please see [10]. If $B$ is singular and $A$ is nonsingular, we have the following result.

Theorem 2**.**

Suppose $A,B\in\mathcal{S}^{n}$ are SDC, $B$ is singular and $A$ is nonsingular. Then

1. (i)

there always exists a nonsingular matrix $U$ such that

[TABLE]

[TABLE]

where $B_{1},A_{1}$ are symmetric of the same size, $B_{1}$ is nonsingular; 2. (ii)

if $A_{3}\succ 0$ then $I_{\succeq}(A,B)=I_{\succeq}(A_{1},B_{1}).$ Otherwise, $I_{\succeq}(A,B)=\emptyset.$

Proof.

(i) Since $B$ is symmetric and singular, there is an orthogonal matrix $Q_{1}$ that puts $B$ into the form

[TABLE]

such that $B_{1}$ is a nonsingular symmetric matrix of size $p\times p,$ where $p={\rm rank}(B).$ Let $\hat{A}:=Q_{1}^{T}AQ_{1}.$ Since $A,B$ are SDC, $\hat{A},\hat{B}$ are SDC too (the converse also holds true). We can write $\hat{A}$ in the following form

[TABLE]

such that $M_{1}$ is a symmetric matrix of size $p\times p,$ $M_{2}$ is a $p\times(n-p)$ matrix, $M_{3}$ is symmetric of size $(n-p)\times(n-p)$ and, importantly, $M_{3}\neq 0.$ Indeed, if $M_{3}=0$ then $\hat{A}=Q_{1}^{T}AQ_{1}=\left(\begin{matrix}M_{1}&M_{2}\\ M_{2}^{T}&0\end{matrix}\right).$ Then we can choose a nonsingular matrix $H$ written in the same partition as $\hat{A}:$ $H=\left(\begin{matrix}H_{1}&H_{2}\\ H_{3}&H_{4}\end{matrix}\right)$ such that both $H^{T}\hat{B}H,H^{T}\hat{A}H$ are diagonal and $H^{T}\hat{B}H$ is of the form

[TABLE]

where $H_{1}^{T}B_{1}H_{1}$ is nonsingular. This implies $H_{2}=0.$ On the other hand,

[TABLE]

is diagonal implying that $H_{1}^{T}M_{2}H_{4}=0,$ and so

[TABLE]

This cannot happen since $\hat{A}$ is nonsingular.

Let $P$ be an orthogonal matrix such that $P^{T}M_{3}P=\texttt{diag}(A_{3},0_{q-r}),$ where $A_{3}$ is a nonsingular diagonal matrix of size $r\times r,r\leq q$ and $p+q=n,$ and set $U_{1}=\texttt{diag}(I_{p},P).$ We then have

[TABLE]

where $\left(\begin{matrix}A_{4}&A_{5}\end{matrix}\right)=M_{2}P,$ $A_{4}$ and $A_{5}$ are of size $p\times r$ and $p\times(q-r),r\leq q,$ respectively. Let

[TABLE]

We can verify that

[TABLE]

and, by (11),

[TABLE]

We denote $A_{1}:=M_{1}-A_{4}A_{3}^{-1}A_{4}^{T}$ and rewrite the matrices as follows

[TABLE]

We now consider whether it can happen that $r<q.$ We note that $U^{T}AU,U^{T}BU$ are SDC. We can choose a nonsingular congruence matrix $K$ written in the form

[TABLE]

such that not only the matrices $K^{T}U^{T}AUK,K^{T}U^{T}BUK$ are diagonal but also $K^{T}U^{T}BUK$ is remained a $p\times p$ nonsingular submatrix at the northwest corner. That is

[TABLE]

is diagonal and $K_{1}^{T}B_{1}K_{1}$ is nonsingular diagonal of size $p\times p.$ This implies that $K_{2}=K_{3}=0.$ Then

[TABLE]

is diagonal implying that

[TABLE]

are diagonal. Note that $U^{T}AU$ is nonsingular, $K_{5}^{T}A_{3}K_{5},K_{6}^{T}A_{3}K_{6}$ must be nonsingular. But then $K_{5}^{T}A_{3}K_{6}=0$ with $A_{3}$ nonsingular is a contradiction. It therefore holds that $q=r.$ Then

[TABLE]

with $B_{1},A_{1},A_{3}$ as desired.

(ii) We note first that $A$ is nonsingular so is $A_{3}.$ If $A_{3}\succ 0,$ then $A+\mu B\succeq 0$ if and only if $A_{1}+\mu B_{1}\succeq 0.$ So it holds in that case $I_{\succeq}(A,B)=I_{\succeq}(A_{1},B_{1}).$ Otherwise, $A_{3}$ is either indefinite or negative definite then $I_{\succeq}(A,B)=\emptyset.$ ∎

The proofs of Theorems 1 and 2 reveal the following important result.

Corollary 1**.**

Suppose $A,B\in\mathcal{S}^{n}$ are SDC and either $A$ or $B$ is nonsingular. Then $I_{\succ}(A,B)$ is nonempty if and only if $I_{\succeq}(A,B)$ has more than one point.

If $A,B$ are both singular, they can be decomposed to either of the following form.

Lemma 3** ([12]).**

For any $A,B\in\mathcal{S}^{n},$ there always exists a nonsingular matrix $U$ that puts $B$ to

[TABLE]

such that $B_{1}$ is nonsingular diagonal of size $p\times p,$ and puts $A$ to $\bar{A}$ of either form

[TABLE]

or

[TABLE]

where $A_{1}$ is symmetric of dimension $p\times p,$ $A_{2}$ is a $p\times(r-s)$ matrix, and $A_{3}$ is a nonsingular diagonal matrix of dimension $s\times s;$ $p,r,s\geq 0,p+r=n.$

It is easy to verify that $A,B$ are SDC if and only if $\bar{A},\bar{B}$ are SDC.

Lemma 4** ([12]).**

1. i)

If $\bar{A}$ takes the form (12) then $\bar{B},\bar{A}$ are SDC if and only if $B_{1},A_{1}$ are SDC and $A_{2}=0;$ 2. ii)

If $\bar{A}$ take the form (13) then $\bar{B},\bar{A}$ are SDC if and only if $B_{1},A_{1}$ are SDC and $A_{2}=0$ or does not exist, i.e., $s=r.$

Now suppose that $A,B$ are SDC, Lemmas 3 and 4 allow to assume without loss of generality that $\bar{B},\bar{A}$ are already SDC. That is

[TABLE]

or

[TABLE]

where $A_{1},B_{1}$ are of the same size and diagonal, $B_{1}$ is nonsingular and if $\bar{A}$ takes the form (12) or (13) and $A_{2}=0$ then $A_{4}={\rm diag}(A_{3},0)$ or if $\bar{A}$ takes the form (13) and $A_{2}$ does not exist then $A_{4}=A_{3}.$ Now we can compute $I_{\succeq}(A,B)$ as follows.

Theorem 3**.**

• (i)

If $\bar{B},\bar{A}$ take the from $(\ref{pt3}),$ then $I_{\succeq}(A,B)=I_{\succeq}(A_{1},B_{1});$

• (ii)

If $\bar{B},\bar{A}$ take the from $(\ref{pt4}),$ then $I_{\succeq}(A,B)=I_{\succeq}(A_{1},B_{1})$ if $A_{4}\succeq 0$ and $I_{\succeq}(A,B)=\emptyset$ otherwise.

We note that $B_{1}$ is nonsingular, $I_{\succeq}(A_{1},B_{1})$ is therefore computed by Theorem 1. Especially, if $I_{\succeq}(A_{1},B_{1})$ has more than one point, then $I_{\succ}(A_{1},B_{1})\neq\emptyset,$ see Corollary 1.

In the rest of this section we consider $I_{\succeq}(A,B)$ when $A,B$ are not SDC. We need first to show that if $A,B$ are not SDC, then $I_{\succeq}(A,B)$ either is empty or has only one point. The proof of Lemma 5 is easy, we omit it.

Lemma 5**.**

If $A,B$ are positive semidefinite then $A,B$ are SDC.

Lemma 6**.**

If $A,B\in\mathcal{S}^{n}$ are not SDC then $I_{\succeq}(A,B)$ either is empty or has only one element.

Proof.

Suppose in contrary that $I_{\succeq}(A,B)$ has more than one elements, then we can choose $\mu_{1},\mu_{2}\in I_{\succeq}(A,B),\mu_{1}\neq\mu_{2}$ such that $C:=A+\mu_{1}B\succeq 0$ and $D:=A+\mu_{2}B\succeq 0.$ By Lemma 5, $C,D$ are SDC, i.e., there is a nonsingular matrix $P$ such that $P^{T}CP,P^{T}DP$ are diagonal. Then $P^{T}BP$ is diagonal because $P^{T}CP-P^{T}DP=(\mu_{1}-\mu_{2})P^{T}BP$ and $\mu_{1}\neq\mu_{2}.$ Since $P^{T}AP=P^{T}CP-\mu_{1}P^{T}BP,$ $P^{T}AP$ is also diagonal. That is $A,B$ are SDC and we get a contradiction. ∎

To know when $I_{\succeq}(A,B)$ is empty or has one element, we need the following result.

Lemma 7** ([13]).**

Let $A,B\in\mathcal{S}^{n},$ B be nonsingular. Let $B^{-1}A$ have the real Jordan normal form $diag(J_{1},\ldots J_{r},J_{r+1},\ldots,J_{m})$, where $J_{1},\ldots,J_{r}$ are Jordan blocks corresponding to real eigenvalues $\lambda_{1},\lambda_{2},\ldots,\lambda_{r}$ of $B^{-1}A$ and $J_{r+1},\ldots,J_{m}$ are Jordan blocks for pairs of complex conjugate roots $\lambda_{i}=a_{i}\pm{\bf i}b_{i},a_{i},b_{i}\in\mathbb{R},i=r+1,r+2,\ldots,m$ of $B^{-1}A$. Then there exists a nonsingular matrix $U$ such that

[TABLE]

[TABLE]

where $\epsilon_{i}=\pm 1,E_{i}=\left(\begin{matrix}0&0&\ldots&0&1\\ 0&0&\ldots&1&0\\ \ldots&\ldots&\ldots&\ldots&\ldots\\ \ldots&\ldots&\ldots&\ldots&\ldots\\ 1&0&\ldots&0&0\end{matrix}\right);$ ${\rm dim}E_{i}={\rm dim}J_{i}=n_{i};n_{1}+n_{2}+\ldots+n_{m}=n.$

Theorem 4**.**

Let $A,B\in\mathcal{S}^{n}$ be as in Lemma 7 and $A,B$ are not SDC. The followings hold.

1. (i)

if $A\succeq 0$ then $I_{\succeq}(A,B)=\{0\};$ 2. (ii)

if $A\nsucceq 0$ and there is a real eigenvalue $\lambda_{l}$ of $B^{-1}A$ such that $A+(-\lambda_{l})B\succeq 0$ then

[TABLE] 3. (iii)

if (i) and (ii) do not occur then $I_{\succeq}(A,B)=\emptyset.$

Proof.

It is sufficient to prove only (iii). Lemma 7 allows us to decompose $A$ and $B$ to the forms (17) and (16), respectively. Since $A,B$ are not SDC, at least one of the following cases must occur.

Case 1 *There is a Jordan block $J_{i}$ such that $n_{i}\geq 2$ and $\lambda_{i}\in\mathbb{R}.$ * We then consider the following principal minor of $A+\mu B:$

[TABLE]

If $n_{i}=2$ then $Y=\epsilon_{i}\left(\begin{matrix}0&\lambda_{i}+\mu\\ \lambda_{i}+\mu&1\end{matrix}\right).$ Since $\mu\neq-\lambda_{i},$ $Y\not\succeq 0$ so $A+\mu B\not\succeq 0.$ If $n_{i}>2$ then $Y$ always contains the following not positive semidefinite principal minor of size $(n_{i}-1)\times(n_{i}-1):$

[TABLE]

So $A+\mu B\not\succeq 0.$

Case 2 There is a Jordan block $J_{i}$ such that $n_{i}\geq 4$ and $\lambda_{i}=a_{i}\pm{\bf i}b_{i}\notin\mathbb{R}.$ We then consider

[TABLE]

This matrix always contains either a principal minor of size $2\times 2:$ $\epsilon_{i}\left(\begin{matrix}b_{i}&a_{i}+\mu\\ a_{i}+\mu&-b_{i}\end{matrix}\right)$ or a principal minor of size $4\times 4:$

[TABLE]

Both are not positive semidefinite for any $\mu\in\mathbb{R}.$ ∎

Similarly, we have the following result. We omit its proof.

Theorem 5**.**

Let $A,B\in\mathcal{S}^{n}$ be not SDC. Suppose $A$ is nonsingular and $A^{-1}B$ has real Jordan normal form $diag(J_{1},\ldots J_{r},J_{r+1},\ldots,J_{m})$, where $J_{1},\ldots,J_{r}$ are Jordan blocks corresponding to real eigenvalues $\lambda_{1},\lambda_{2},\ldots,\lambda_{r}$ of $A^{-1}B$ and $J_{r+1},\ldots,J_{m}$ are Jordan blocks for pairs of complex conjugate roots $\lambda_{i}=a_{i}\pm{\bf i}b_{i},a_{i},b_{i}\in\mathbb{R},i=r+1,r+2,\ldots,m$ of $A^{-1}B$.

1. (i)

If $A\succeq 0$ then $I_{\succeq}(A,B)=\{0\};$ 2. (ii)

If $A\nsucceq 0$ and there is a real eigenvalue $\lambda_{l}\neq 0$ of $A^{-1}B$ such that $A+\left(-\dfrac{1}{\lambda_{l}}\right)B\succeq 0$ then $I_{\succeq}(A,B)=\left\{-\dfrac{1}{\lambda_{l}}\right\};$ 3. (iii)

If cases $(i)$ and $(ii)$ do not occur then $I_{\succeq}(A,B)=\emptyset.$

Finally, if $A$ and $B$ are not SDC and both singular. Lemma 3 indicates that $A$ and $B$ can be simultaneously decomposed to $\bar{A}$ and $\bar{B}$ in either (12) or (13). If $\bar{A}$ and $\bar{B}$ take the forms (12) and $A_{2}=0$ then $I_{\succeq}(A,B)=I_{\succeq}(A_{1},B_{1}),$ where $A_{1},B_{1}$ are not SDC and $B_{1}$ is nonsingular. In this case we apply Theorem 4 to compute $I_{\succeq}(A_{1},B_{1}).$ If $\bar{A}$ and $\bar{B}$ take the forms (13) and $A_{2}=0.$ In this case, if $A_{3}$ is not positive definite then $I_{\succeq}(A,B)=\emptyset.$ Otherwise, $I_{\succeq}(A,B)=I_{\succeq}(A_{1},B_{1}),$ where $A_{1},B_{1}$ are not SDC and $B_{1}$ is nonsingular, again we can apply Theorem 4. Therefore we need only to consider the case $A_{2}\neq 0$ with noting that $I_{\succeq}(A,B)\subset I_{\succeq}(A_{1},B_{1}).$

Theorem 6**.**

Given $A,B\in\mathcal{S}^{n}$ are not SDC and singular such that $\bar{A}$ and $\bar{B}$ take the forms in either (12) or (13) with $A_{2}\neq 0.$ Suppose that $I_{\succeq}(A_{1},B_{1})=[a,b],a<b.$ Then, if $a\not\in I_{\succeq}(A,B)$ and $b\not\in I_{\succeq}(A,B)$ then $I_{\succeq}(A,B)=\emptyset.$

Proof.

We consider $\bar{A}$ and $\bar{B}$ in (13), the form in (12) is considered similarly. Suppose in contrary that $I_{\succeq}(A,B)=\{\mu_{0}\}$ and $a<\mu_{0}<b.$ Since $I_{\succeq}(A_{1},B_{1})$ has more than one point, by Lemma 6, $A_{1},B_{1}$ are SDC. Let $Q_{1}$ be a $p\times p$ nonsingular matrix such that $Q_{1}^{T}A_{1}Q_{1},Q_{1}^{T}B_{1}Q_{1}$ are diagonal, then $Q^{T}_{1}(A_{1}+\mu_{0}B_{1})Q_{1}:=\rm{diag}(\gamma_{1},\gamma_{2},\ldots,\gamma_{p})$ is a diagonal matrix. Moreover, $B_{1}$ is nonsingular, we have $I_{\succ}(A_{1},B_{1})=(a,b),$ please see Corollary 1. Then $\gamma_{i}>0$ for $i=1,2,\ldots,p$ because $\mu_{0}\in I_{\succ}(A_{1},B_{1}).$ Let $Q:=\rm{diag}(Q_{1},I_{s},I_{r-s})$ we then have

[TABLE]

We note that $I_{\succeq}(A,B)=\{\mu_{0}\}$ is singleton implying ${\rm det}(A+\mu_{0}B)=0$ and so ${\rm det}(Q^{T}(\bar{A}+\mu_{0}\bar{B})Q)=0.$ On the other hand, since $A_{3}$ is nonsingular diagonal and $A_{1}+\mu_{0}B_{1}\succ 0,$ the first $p+s$ columns of the matrix $Q^{T}(\bar{A}+\mu_{0}\bar{B})Q$ are linearly independent. One of the following cases must occur: i) the columns of the right side submatrix $\left(\begin{matrix}Q^{T}_{1}A_{2}\\ 0_{s\times(r-s)}\\ 0_{(r-s)\times(r-s)}\end{matrix}\right)$ are linearly independent and at least one column, suppose $(c_{1},c_{2},\ldots,c_{p},0,0,\ldots,0)^{T},$ is a linear combination of the columns of the matrix

[TABLE]

where ${\rm column_{i}}$ is the $i$th column of the matrix or ii) the columns of the right side submatrix $\left(\begin{matrix}Q^{T}_{1}A_{2}\\ 0_{s\times(r-s)}\\ 0_{(r-s)\times(r-s)}\end{matrix}\right)$ are linearly dependent. If the case i) occurs then there are scalars $a_{1},a_{2},\ldots,a_{p}$ which are not all zero such that

[TABLE]

Equation (18) implies that

[TABLE]

which further implies

[TABLE]

This cannot happen with $\gamma_{i}>0$ and $(a_{1})^{2}+(a_{2})^{2}+\ldots+(a_{p})^{2}\neq 0.$ This contradiction shows that $I_{\succeq}(A,B)=\emptyset.$ If the case ii) happens then there always exists a nonsingular matrix $H$ such that

[TABLE]

where $\hat{A}_{2}$ is a full column-rank matrix. Let

[TABLE]

we have $I_{\succeq}(A,B)=I_{\succeq}(\bar{A},\bar{B})=I_{\succeq}(\hat{A},\hat{B})$ and so $I_{\succeq}(\hat{A},\hat{B})=\{\mu_{0}\}.$ This implies ${\rm det}(\hat{A}+\mu_{0}\hat{B})=0,$ and the right side submatrix $\left(\begin{matrix}\hat{A}_{2}\\ 0\\ 0\end{matrix}\right)$ is full column-rank. We return to the case i). ∎

3 Application for the GTRS

In this section we find an optimal Lagrangian multiplier

[TABLE]

together with an optimal solution $x^{*}$ for the GTRS (1). We need first to recall the following optimality conditions for the GTRS (1).

Lemma 8** ([10]).**

A vector $x^{*}\in\mathbb{R}^{n}$ is an optimal solution to GTRS (1) if and only if there exists $\mu^{*}\geq 0$ such that

[TABLE]

In fact, the conditions (19) and (22) are necessary and sufficient for the GTRS (1) to be bounded below [14]. However, a bounded GTRS may have no optimal solution, see, for example [1]. The conditions (20)-(21) are thus added to guarantee the existence of an optimal solution to GTRS (1). To check whether a $\mu\in I$ satisfies conditions (19)-(21) we need to apply the following result.

Lemma 9** ([10]).**

Suppose $I_{PD}:=I_{\succ}(A,B)\cap[0,\infty)$ is nonempty. If $x(\mu)$ is the solution of (19), and if the function $\varphi:\mathbb{R}\rightarrow\mathbb{R}$ is defined on $I_{PD}$ by

[TABLE]

then $\varphi$ is strictly decreasing on $I_{PD}$ unless $x(\cdot)$ is constant on $I_{PD}.$

We note that $I$ can be a nonempty interval while $I_{PD}$ is empty. Fortunately, our Corollary 1 shows that this case happens only when $A$ and $B$ are both singular. Then we can decompose $A,B$ to $\bar{A},\bar{B}$ and apply Theorem 3 if $A,B$ are SDC, Theorem 4 or Theorem 5 or Theorem 6 if $A,B$ are not SDC.

By Lemma 8, if $I=\emptyset,$ the GTRS (1) has no optimal solution, it is even unbounded from below [14]. If $I\neq\emptyset$ but has only one element: $I=I_{\succeq}(A,B)\cap[0,+\infty)=\{\mu\}.$ Then we need only to solve the linear system (19)-(21) to check whether $\mu^{*}=\mu.$ If $I$ is an interval, suppose $I=[\mu_{1},\mu_{2}],$ where $\mu_{1}\geq 0$ and $\mu_{2}$ may be $\infty,$ then $A,B$ are SDC. We assume that $A={\rm diag}(\alpha_{1},\alpha_{2},\ldots,\alpha_{n}),B={\rm diag}(\beta_{1},\beta_{2},\ldots,\beta_{n}).$ The equation (19) is then of the following simple form

[TABLE]

Solving (23) for a fixed $\mu$ is very simple, the main duty is thus finding $\mu^{*}$ in the following cases.

If at least one of the matrices $A$ and $B$ is nonsingular then $I_{PD}\neq\emptyset$ and $I={\rm closure}(I_{PD}).$ The GTRS (1) then attains a unique optimal solution $x^{*}$ at an optimal Lagrange multiplier $\mu^{*}$ [10]. We first check whether $\mu^{*}=0.$ If not, we apply Lemma 9 for finding $\mu^{*}$ such that $\varphi(\mu^{*})=0.$ Observe from (23) that $x(\mu)$ is constant on $I_{PD}$ only when $\beta_{i}=b_{i}=0$ for all $i=1,2,\ldots,n.$ This case can be dealt with easily since $g(x)$ is then constant. Otherwise, $\varphi(\mu)$ is strictly decreasing on $I_{PD}$ and we have the following results.

Lemma 10** ([4],[1]).**

Suppose the Slater condition holds for the GTRS (1), i.e., there exists $\bar{x}\in\mathbb{R}^{n}$ such that $g(\bar{x})<0,$ and $I_{PD}\neq\emptyset.$

1. (a)

If $\varphi(\mu)>0$ on $I_{PD}$ and $\mu_{2}<\infty,$ then $\mu^{*}=\mu_{2};$ 2. (b)

If $\varphi(\mu)<0$ on $I_{PD}$ then $\mu^{*}=\mu_{1};$ 3. (c)

If $\varphi(\mu)$ changes its sign on $I_{PD}$ then $\mu^{*}\in I_{PD};$ 4. (d)

If $\varphi(\mu)>0$ on $I_{PD}$ then $\mu_{2}$ cannot be $\infty.$

The case (d) indicates that if $I=[\mu_{1},\infty)$ and $\varphi(\mu_{1})>0$ then $\mu_{1}<\mu^{*}<\infty.$ 2. 2.

If both $A$ and $B$ are singular, by Lemma 4, $B,A$ are decomposed to the form either (14) or (15), and are now in the form

[TABLE]

or

[TABLE]

where $\beta_{1},\beta_{2},\ldots,\beta_{p}$ are nonzero. We have

[TABLE]

where $B_{1}=\texttt{diag}(\beta_{1},\beta_{2},\ldots,\beta_{p}),A_{1}=\texttt{diag}(\alpha_{1},\alpha_{2},\ldots,\alpha_{p}),$ please see Theorem 3 and Corollary 1. We note also that if $\alpha_{i}>0$ for $i=p+1,\ldots,n,$ then $I_{\succ}(A,B)=I_{\succ}(A_{1},B_{1})$ and we can apply Lemma 10. Otherwise, $I_{\succ}(A,B)=\emptyset$ and the GTRS (1) may have no optimal solution. We deal with this case as follows.

If $A,B$ take the form (24), the equations (23) become

[TABLE]

If $a_{i}=b_{i}=0$ for $i=p+1,\ldots,n,$ then (1) is reduced to a GTRS of $p$ variables with matrices $A_{1},B_{1}$ such that $I_{\succ}(A_{1},B_{1})\neq\emptyset.$ We then apply Lemma 10 for it. Otherwise, either (26) has no solution $x$ for all $\mu\in I$ or it has solutions at only one $\mu\in I.$ Then we check easily whether $\mu^{*}=\mu.$

If $A,B$ take the form (25), the equations (23) become

[TABLE]

By the same arguments as above, either (1) is then reduced to a GTRS of $p+s$ variables with matrices

[TABLE]

such that $I_{\succ}(\bar{A}_{1},\bar{B}_{1})\neq\emptyset.$ We then apply Lemma 10 for it. Otherwise, either (27) has no solution $x$ for all $\mu\in I$ or it has solutions at only one $\mu\in I.$

4 Conclusion and remarks

In this paper, we showed that for a given pair of real symmetric matrices $A,B,$ the set $I_{\succeq}(A,B)$ of real values $\mu\in\mathbb{R}$ such that the matrix pencil $A+\mu B$ is positive semidefinite is always computable by solving the generalized eigenvalue problem of $n\times n$ dimension. The computation is considered in two separated cases: If $A,B$ are not simultaneously diagonalizable via congruence (SDC), $I_{\succeq}(A,B)$ is either empty or singleton, while if $A,B$ are SDC, $I_{\succeq}(A,B)$ can be empty, singleton or an interval. In case $I_{\succeq}(A,B)$ is an interval, if either $A$ or $B$ is nonsingular, $I_{\succeq}(A,B)$ is the closure of the positive definite interval $I_{\succ}(A,B).$ Otherwise, $A,B$ are decomposable to block diagonals of submatrices $A_{1},B_{1}$ with $B_{1}$ nonsingular such that $I_{\succeq}(A,B)$ is now the closure of $I_{\succ}(A_{1},B_{1}).$ With $I_{\succeq}(A,B)$ in hand, we are able to solve the generalized trust region subproblem (1) not only in the easy-case when $I_{\succ}(A,B)$ is nonempty but also in the hard-case by only solving the linear equations. Our result needs only to solve the generalized eigenvalue problem of $n\times n$ dimension compared with an $(2n+1)\times(2n+1)$ generalized eigenvalue problem in [1]. On the other hand, we can completely deal with the hard-case of the GTRS (1), which was an open problem in [10] and [8].