An extension of Yuan's Lemma and its applications in optimization

TL;DR

This paper extends Yuan's Lemma to multiple matrices with rank constraints, enabling broader second-order optimality conditions in constrained optimization problems, especially with quadratic constraints.

Contribution

It generalizes Yuan's Lemma to more matrices with rank at most 2, relaxing previous boundedness assumptions on Lagrange multipliers.

Findings

Extended Yuan's Lemma to multiple matrices with rank constraints

Generalized second-order optimality conditions for quadratic constrained problems

Applicable to problems without constant rank of the Jacobian

Abstract

We prove an extension of Yuan's Lemma to more than two matrices, as long as the set of matrices has rank at most 2. This is used to generalize the main result of [A. Baccari and A. Trad. On the classical necessary second-order optimality conditions in the presence of equality and inequality constraints. SIAM J. Opt., 15(2):394--408, 2005], where the classical necessary second-order optimality condition is proved under the assumption that the set of Lagrange multipliers is a bounded line segment. We prove the result under the more general assumption that the hessian of the Lagrangian evaluated at the vertices of the Lagrange multiplier set is a matrix set with at most rank 2. We apply the results to prove the classical second-order optimality condition to problems with quadratic constraints and without constant rank of the jacobian matrix.

Equations

$$\underset{i=1,\dots ,m}{\max }\{x^{\mathtt{T}}{A}_{i}x\}\ge 0,\forall x\in K,$$

$$\exists t\in {\Lambda }_m\text{mbox}suchthat\sum _{i=1}^m{t}_i{A}_i\text{mbox}ispositivesemidefiniteonK.$$

$$A_{1}=\left(\begin{array}[]{cc}1&-1\\ -1&1\end{array}\right),A_{2}=\left(\begin{array}[]{cc}-2&1\\ 1&1\end{array}\right),A_{3}=\left(\begin{array}[]{cc}4&-3\\ -3&1\end{array}\right).$$

$$A_{1}=\left(\begin{array}[]{cc}-1&0\\ 0&1\end{array}\right),A_{2}=\left(\begin{array}[]{cc}1&2\\ 2&-2\end{array}\right),A_{3}=\left(\begin{array}[]{cc}0&-2\\ -2&1\end{array}\right).$$

$$\begin{array}[]{ll}\mbox{Minimize}&f(x),\\ \mbox{subject to}&h(x)=0,\\ &g(x)\leq 0,\end{array}$$

$$x\mapsto L(x,\lambda ,\mu )=f(x)+\sum _{i=1}^{p_1}{\lambda }_i{h}_i(x)+\sum _{i=1}^{p_2}{\mu }_i{g}_i(x),$$

$$\sum _{i=1}^{p_1}{\alpha }_i\nabla h_i(x^{\ast })+\sum _{i\in A(x^{\ast })}{\beta }_i\nabla g_i(x^{\ast })=0,{\alpha }_i\in \mathbb{R},i=1,\dots ,p_1;{\beta }_i\ge 0,i\in A(x^{\ast }).$$

$$\Lambda (x^{\ast })=\{(\lambda ,\mu )\in {\mathbb{R}}^{p_1}\times {\mathbb{R}}_{+}^{p_2}\mid \nabla L(x^{\ast },\lambda ,\mu )=0,{\mu }^{\mathtt{T}}g(x^{\ast })=0\}.$$

$$d^{\mathtt{T}}{\nabla }^{2}L(x^{\ast },\lambda ,\mu )d\ge 0,$$

$$C(x^{*})=\left\{\begin{array}[]{ll}d\in{\mathbb{R}}^{n}:&\nabla h_{i}(x^{*})^{\mathtt{T}}d=0,i=1,\dots,p_{1};\nabla g_{i}(x^{*})^{\mathtt{T}}d\leq 0,i\in A(x^{*});\\ &\nabla f(x^{*})^{\mathtt{T}}d=0.\end{array}\right\}$$

$$d^{\mathtt{T}}{\nabla }^{2}L(x^{\ast },\lambda ,\mu )d\ge 0,\forall d\in K.$$

$$\underset{(\lambda ,\mu )\in \Lambda (x^{\ast })}{\max }\{d^{\mathtt{T}}{\nabla }^{2}L(x^{\ast },\lambda ,\mu )d\}\ge 0,\forall d\in K.$$

$$\underset{i=1,\dots ,v}{\max }\{d^{\mathtt{T}}{\nabla }^{2}L(x^{\ast },{\lambda }^i,{\mu }^i)d\}\ge 0,\forall d\in K.$$

$$\begin{array}[]{ll}\mbox{Minimize}&z,\\ \mbox{subject to}&g_{i}(x,z):=\frac{1}{2}x^{\mathtt{T}}A_{i}x-z\leq 0,i=1,\dots,m,\end{array}$$

$$\forall x\in {\mathbb{R}}^n,{\alpha }_x(A-C)x+{\beta }_x(B-C)x=0,\text{mbox}forsome({\alpha }_x,{\beta }_x)\ne (0,0).$$

$${\alpha }_1(A-C)v_1+{\beta }_1{\lambda }_1{v}_1=0,\text{mbox}forsome({\alpha }_1,{\beta }_1)\ne (0,0).$$

$${\alpha }_1{v}_1^{\mathtt{T}}(A-C)v_1+{\beta }_1{\lambda }_1=0\text{mbox}and{\alpha }_1{v}_i^{\mathtt{T}}(A-C)v_1=0,i>1.$$

$$v_1^{\mathtt{T}}(A-C)v_1=-\frac{{\beta }_1}{{\alpha }_1}{\lambda }_1\text{mbox}and{v}_i^{\mathtt{T}}(A-C)v_1=0,i>1.$$

$${\alpha }_i(A-C)v_1+{\alpha }_i(A-C)v_i+{\beta }_i({\lambda }_1{v}_1+{\lambda }_i{v}_i)=0.$$

$$-{\alpha }_i\frac{{\beta }_1}{{\alpha }_1}{\lambda }_1+{\beta }_i{\lambda }_1=0,$$

$$(A-C)u_i+\delta (B-C)u_i=0,\text{mbox}foralli=1,\dots ,n,$$

Full text

**Keywords: **Quadratic forms, second-order optimality conditions, global convergence.

1 Introduction

In a general nonlinear optimization problem, it is well known that when the set of Lagrange multipliers is bounded, a local minimizer satisfies a second-order optimality condition in the form of the maximum of many quadratic forms being non-negative in the critical cone, where each quadratic form is defined by the Hessian of the Lagrangian function evaluated at different Lagrange multipliers.

In this paper we are interested in second-order optimality conditions that can be verified with one single Lagrange multiplier. This is motivated by algorithmic considerations, since algorithms usually generate a primal-dual sequence and one is interested in proving that limit points of this sequence satisfy a first- or second-order optimality condition. Under this setting, one is usually restricted to considering a smaller subset of the true critical cone.

Yuan’s Lemma [23] gives an important tool for this type of results, since it states that when the maximum of two quadratic forms is non-negative, some convex combination of the matrices is positive semidefinite. This implies that when the set of Lagrange multipliers is a bounded line segment, one can find a Lagrange multiplier such that the Hessian of the Lagrangian is positive semidefinite on a subset of the critical cone. This is the main result of [5]. However, this approach does not work when there are more than two quadratic forms, that is, when the Lagrange multiplier set is larger than a line segment.

In Section 2, we will extend Yuan’s Lemma to more than two quadratic forms. This will be done under an assumption of redundancy on the matrices defining the quadratic forms, in the sense that it is a matrix set of rank at most $2$. This gives, in Section 3, an optimality condition verifiable on one single Lagrange multiplier under an assumption of redundancy of the set of Hessians of the Lagrangian matrices that includes as a particular case the case of a Lagrange multiplier set equals to a line segment. In Section 4 we apply the results to a special form of quadratically-constrained problems, and we prove that our assumption holds when the rank of the Jacobian matrix increases at most by one in the neighborhood of a local solution. Section 5 gives some conclusions.

Notation: A set $K\subseteq{\mathbb{R}}^{n}$ is a first-order cone if $K$ is the direct sum of a subspace and a ray, where a ray is a set of the form $\{td_{0}\mid t\geq 0\}$, for some $d_{0}\in{\mathbb{R}}^{n}$. Given a set $K\subseteq{\mathbb{R}}^{n}$ and a symmetric matrix $A\in{\mathbb{R}}^{n\times n}$, we say that $A$ is positive semidefinite on $K$ if $x^{\mathtt{T}}Ax\geq 0$ for all $x\in K$. We denote by ${\mathbb{R}}^{m}_{+}$ the non-negative orthant of ${\mathbb{R}}^{m}$ and $\Lambda_{m}=\{t\in{\mathbb{R}}^{m}_{+}\mid\sum_{i=1}^{m}t_{i}=1\}$ the $m$-dimensional simplex. The rank of a set of matrices is the maximum number of linearly independent matrices viewed as vectors on the appropriate space.

2 Extension of Yuan’s Lemma

An important result related to the trust-region subproblem is the following:

Lemma 1** (Yuan’s Lemma [23]).**

Let $A,B\in{\mathbb{R}}^{n\times n}$ be symmetric matrices and $K\subseteq{\mathbb{R}}^{n}$ be a first-order cone. Then the following conditions are equivalent:

• •

$\max\{d^{\mathtt{T}}Ad,d^{\mathtt{T}}Bd\}\geq 0,\forall d\in K$,

• •

there exists $t_{1}\geq 0,t_{2}\geq 0$ with $t_{1}+t_{2}=1$ such that $t_{1}A+t_{2}B$ is positive semidefinite on $K$.

Lemma 1 is known as Yuan’s Lemma [23] and it has had many important generalizations [5, 13, 16, 10, 9]. In particular, our presentation of Yuan’s Lemma corresponds to [5, Corollary 3.2]. Yuan’s Lemma when $K={\mathbb{R}}^{n}$ can be easily proved with a separation argument due to the convexity of the set $\{(x^{\mathtt{T}}Ax,x^{\mathtt{T}}Bx)\mid x\in{\mathbb{R}}^{n}\}\subseteq{\mathbb{R}}^{2}$ given in [11]. See also some related convexity results in [8, 20, 21, 19, 22].

In the next result we extend Yuan’s Lemma to $m$ matrices $A_{1},\dots,A_{m}\in{\mathbb{R}}^{n\times n}$ with $\mbox{rank}(\{A_{1},\dots,A_{m}\})\leq 2$. A similar assumption is used to obtain a result of S-Lemma type in [19, Proposition 3.5], however, our proof is elementary and our result is more general in the sense that it holds for first-order cones.

Lemma 2**.**

Let $A_{i}\in{\mathbb{R}}^{n\times n},i=1,\dots,m,m\geq 2$, be such that $A_{i}=\alpha_{i}A_{1}+\beta_{i}A_{2},i=3,\dots,m$ for some $(\alpha_{i},\beta_{i})\in{\mathbb{R}}^{2},i=3,\dots,m$ and $K\subseteq{\mathbb{R}}^{n}$ a first-order cone. Then, the following are equivalent:

[TABLE]

[TABLE]

Proof.

The fact that (2.2) implies (2.1) can be easily seen by contradiction. The reciprocal implication is done by induction on $m$. If $m=2$, the result follows from Lemma 1. Assume the assertion is true for $m-1\geq 2$. Assume (2.1). Since $A_{m}=\alpha_{m}A_{1}+\beta_{m}A_{2}$, let us consider the following cases:

The case $(\alpha_{m},\beta_{m})=(0,0)$ is trivial. If $\alpha_{m}\geq 0$ and $\beta_{m}\geq 0$, $(\alpha_{m},\beta_{m})\neq(0,0)$, then $x^{\mathtt{T}}A_{1}x<0$ and $x^{\mathtt{T}}A_{2}x<0$ imply $x^{\mathtt{T}}A_{m}x<0$. Thus, (2.1) implies $\max_{i=1,\dots,m-1}\{x^{\mathtt{T}}A_{i}x\}\geq 0,\forall x\in K$. If $\alpha_{m}<0$ and $\beta_{m}>0$, we have $x^{\mathtt{T}}A_{1}x<0$ and $x^{\mathtt{T}}A_{m}x<0$ imply $x^{\mathtt{T}}A_{2}x<-\frac{\alpha_{m}}{\beta_{m}}x^{\mathtt{T}}A_{1}x<0$, hence, (2.1) implies $\max_{i=1,\dots,m,i\neq 2}\{x^{\mathtt{T}}A_{i}x\}\geq 0,\forall x\in K$. It is clear that any matrix $A_{i},i=1,\dots,m,i\neq 2$ can be written as the linear combination of two fixed matrices in this set. The case $\alpha_{m}<0$ and $\beta_{m}=0$ gives $\max\{x^{\mathtt{T}}A_{1}x,x^{\mathtt{T}}A_{m}x\}\geq 0,\forall x\in K$. Also, the case $\alpha\geq 0,\beta<0$ is analogous. In any of the above cases, the result follows from Lemma 1 or the inductive assumption. If $\alpha_{m}<0$ and $\beta_{m}<0$, we have $t_{1}A_{1}+t_{2}A_{2}+t_{m}A_{m}=(t_{1}+\alpha_{m}t_{m})A_{1}+(t_{2}+\beta_{m}t_{m})A_{2}=0$, which is positive semidefinite on $K$, for $(t_{1},t_{2},t_{m})=(\frac{-\alpha_{m}}{1-\alpha_{m}-\beta_{m}},\frac{-\beta_{m}}{1-\alpha_{m}-\beta_{m}},\frac{1}{1-\alpha_{m}-\beta_{m}})\in\Lambda_{3}$.

∎

Example 1**.**

Let

[TABLE]

Note that $A_{3}=2A_{1}-A_{2}$. A simple calculation shows that $\max_{i=1,2,3}\{x^{\mathtt{T}}A_{i}x\}\geq 0$ for all $x$, and, as predicted by Lemma 2, $t_{1}A_{1}+t_{2}A_{2}+t_{3}A_{3}$ is positive semidefinite, for instance, for $(t_{1},t_{2},t_{3})=(0,\frac{3}{5},\frac{2}{5})$. As it is done in the proof, this is a consequence of the condition $\max\{x^{\mathtt{T}}A_{2}x,x^{\mathtt{T}}A_{3}x\}\geq 0$ for all $x$ that necessarily holds.

Example 2**.**

Let

[TABLE]

Note that $A_{3}=-A_{1}-A_{2}$. Therefore, $\frac{1}{3}A_{1}+\frac{1}{3}A_{2}+\frac{1}{3}A_{3}=0$ is positive semidefinite and clearly it can not exist $x\in{\mathbb{R}}^{2}$ with $x^{\mathtt{T}}A_{1}x<0$, $x^{\mathtt{T}}A_{2}x<0$ and $x^{\mathtt{T}}A_{3}x<0$, which implies the maximum of the three quadratic forms is non-negative. Note, however, that for all $i\neq j$, it is not the case that $\max\{x^{\mathtt{T}}A_{i}x,x^{\mathtt{T}}A_{j}x\}\geq 0$ for all $x$.

3 A Second-order Optimality Condition

Let us consider the nonlinear optimization problem

[TABLE]

where $f:{\mathbb{R}}^{n}\to{\mathbb{R}},h:{\mathbb{R}}^{n}\to{\mathbb{R}}^{p_{1}},g:{\mathbb{R}}^{n}\to{\mathbb{R}}^{p_{2}}$ are twice continuously differentiable functions. For a feasible $x$, we define $A(x)=\{i\in\{1,\dots,p_{2}\}\mid g_{i}(x)=0\}$, the index set of active inequality constraints. Given $(\lambda,\mu)\in{\mathbb{R}}^{p_{1}}\times{\mathbb{R}}^{p_{2}}_{+}$, we consider the Lagrangian function

[TABLE]

with gradient $\nabla L(x,\lambda,\mu)$ and Hessian $\nabla^{2}L(x,\lambda,\mu)$, where derivative is taken with respect to $x$. We will assume that at a local minimizer $x^{*}$ of (3.1), the Mangasarian-Fromovitz constraint qualification holds:

Definition 1**.**

A feasible point $x^{*}$ of problem (3.1) satisfies the Mangasarian-Fromovitz constraint qualification [15] when $(\alpha,\beta)=(0,0)$ is the only solution of

[TABLE]

It is well known [12] that at a local minimizer $x^{*}$, the Mangasarian-Fromovitz constraint qualification is equivalent to the non-emptyness and boundedness of the Lagrange multiplier set

[TABLE]

We observe also that $\Lambda(x^{*})$ is always a closed polyhedron. The following optimality condition is well-known:

Theorem 1** ([7]).**

Let $x^{*}$ be a local minimizer of (3.1) satisfying the Mangasarian-Fromovitz constraint qualification. Then, for each $d\in C(x^{*})$, there is a Lagrange multiplier $(\lambda,\mu)\in\Lambda(x^{*})$ satisfying

[TABLE]

where

[TABLE]

is the critical cone.

Theorem 1 is the classical second-order necessary optimality condition under Mangasarian-Fromovitz constraint qualification alone. A version of it without constraint qualifications and using Fritz-John multipliers is known as a no-gap optimality condition in terms of sufficiency, see details in [7].

Note that condition (3.2) holds with different Lagrange multipliers for different directions in the critical cone. In this paper we are interested in conditions that ensure the validity of (3.2) for the same Lagrange multiplier $(\lambda,\mu)\in\Lambda(x^{*})$, even if for that we need to consider a smaller subset of the true critical cone $C(x^{*})$. This is motivated by algorithmic considerations, since algorithms usually generate only one Lagrange multiplier approximation. See the discussion in [6].

In the next theorem, we will use Lemma 2 to formulate a new optimality condition of this type. It is a generalization of the main result of [5], where $\Lambda(x^{*})$ was assumed to be a bounded line segment.

Theorem 2**.**

Let $x^{*}$ be a local minimizer of (3.1) satisfying the Mangasarian-Fromovitz constraint qualification. Let $(\lambda^{1},\mu^{1}),\dots,(\lambda^{v},\mu^{v})$ be the vertices of the Lagrange multiplier set $\Lambda(x^{*})$. If the rank of $\{\nabla^{2}L(x^{*},\lambda^{i},\mu^{i})\}_{i=1}^{v}$ is at most $2$, then for every first-order cone $K\subseteq C(x^{*})$, there exists $(\lambda,\mu)\in\Lambda(x^{*})$ such that

[TABLE]

Proof.

Let $K\subseteq C(x^{*})$ be a first-order cone. From Theorem 1, we have

[TABLE]

Since Mangasarian-Fromovitz constraint qualification implies that $\Lambda(x^{*})$ is a non-empy, compact, polyhedral set, let $(\lambda^{1},\mu^{1}),\dots,(\lambda^{v},\mu^{v})$ be its vertices. Since for each $d\in K$ the maximization problem (3.5) is a linear programming with non-empty and compact feasible region, its solution is attained at a vertex, which implies

[TABLE]

Lemma 2 gives $t\in\Lambda_{v}$ such that $\sum_{i=1}^{v}t_{i}\nabla^{2}L(x^{*},\lambda^{i},\mu^{i})$ is positive semidefinite on $K$, and the linearity of $\nabla^{2}L(x^{*},\cdot,\cdot)$ gives $\nabla^{2}L(x^{*},\lambda,\mu)$ positive semidefinite on $K$ for $(\lambda,\mu)=\sum_{i=1}^{v}t_{i}(\lambda^{i},\mu^{i})\in\Lambda(x^{*})$.

∎

Note that when the critical cone is a first-order cone, Theorem 2 gives a full second-order optimality condition in terms of the true critical cone. One condition ensuring this, is the generalized scrict complementarity slackness, namely, that there is at most one index $i_{0}\in A(x^{*})$ such that $\mu_{i_{0}}=0$ for all $(\lambda,\mu)\in\Lambda(x^{*})$. See [5].

Let us revisit Examples 1 and 2. The fact that $\max_{i=1,2,3}\{x^{\mathtt{T}}A_{i}x\}\geq 0$ for all $x$, implies that $x^{*}=(0,0),z^{*}=0$ is a global minimizer of the problem of minimizing $z$, subject to $\frac{1}{2}x^{\mathtt{T}}A_{i}x-z\leq 0,i=1,2,3$. It is easy to see that the Mangasarian-Fromovitz constraint qualification holds, the set of Lagrange multipliers at $(x^{*},z^{*})$ is the simplex $\Lambda_{3}$ and the critical cone is ${\mathbb{R}}^{2}\times\{0\}$. Note that the quadratic form defined by the Hessian of the Lagrangian at $(x^{*},z^{*})$ evaluated at each of the three vertices of $\Lambda_{3}$ and restricted to the critical cone gives the quadratic forms defined by the three matrices $A_{1},A_{2}$ and $A_{3}$. Hence, the optimality condition of Theorem 2 translates precisely to the fact that $t_{1}A_{1}+t_{2}A_{2}+t_{3}A_{3}$ is positive semidefinite for some $(t_{1},t_{2},t_{3})\in\Lambda_{3}$, which holds as observed in Examples 1 and 2. In the next session we will give a sufficient condition for the rank assumption to hold under this setting.

4 Application to a Quadratically-constrained Problem

It is well known that without the rank assumption, the optimality condition of Theorem 2 does not hold. Known counter-examples [3, 2, 4] are of the form

[TABLE]

where $A_{i}\in{\mathbb{R}}^{n\times n},i=1,\dots,m$, are symmetric matrices and $(x^{*},z^{*})=(0,0)\in{\mathbb{R}}^{n}\times{\mathbb{R}}$ is a solution. This is a special case of the well known quadratically-constrained quadratic programming problem, a well-studied difficult non-convex optimization problem [14, 18, 17]. Note that the linear objective can replace an objective function $f(x)$ by adding the constraint $f(x)-z\leq 0$. Note that the critical cone $C(x^{*},z^{*})$ is the subspace ${\mathbb{R}}^{n}\times\{0\}$. We will prove that for this type of problem, in order to fulfill our rank assumption, it is sufficient to assume that the rank of the Jacobian matrix (that is, the matrix of gradients of active constraints at the solution) increases at most by one in a neighborhood of the solution. It is conjectured in [1] that for the general problem (3.1) satisfying Mangasarian-Fromovitz constraint qualification at the local solution $x^{*}$, when the rank of the Jacobian matrix increases at most by one in a neighborhood of the solution, the optimality condition (3.4) holds with the cone $K$ equal to the lineality space of $C(x^{*})$. This is known to hold when the rank of the Jacobian matrix is constant [1], one less than the number of active constraints [5], one less than the dimension $n$ [6], or when the Jacobian matrix has a smooth singular value decomposition [6]. We refer the reader to [6] for a discussion about this conjecture and related results. Our result proves this conjecture for the class of quadratically-constrained problems (4.1). Note that this is not a particular case of any known case where the conjecture holds. In particular, [6, Example 3.3] shows a problem of type (4.1) with $m=3$ active constraints and $n=3$ variables where the singular value decomposition is not smooth, and the rank of the Jacobian matrix increases from $1$ to at most $2$ in a neighborhood of the origin. Our theorem applies to this problem, whereas no other known result applies to obtain (3.4). To present our result, we start with the following lemma:

Lemma 3**.**

Given $a\neq 0$ and $A_{i}\in{\mathbb{R}}^{n\times n}$ symmetric matrices, $i=1,\dots,m$, define v_{i}(x)=\left(\begin{array}[]{c}A_{i}x\\ a\end{array}\right)\in{\mathbb{R}}^{n+1},i=1,\dots,m and $J(x)\in{\mathbb{R}}^{(n+1)\times m}$ given column-wise by $J(x)=[v_{1}(x)\cdots v_{m}(x)]$. If the rank of $J(x)$ is at most $2$ for all $x\in{\mathbb{R}}^{n}$, then the rank of $\{A_{1},\dots,A_{m}\}$ is at most $2$.

Proof.

Take any three distinct indexes $\{i_{1},i_{2},i_{3}\}\subseteq\{1,\dots,m\}$ and let $A=A_{i_{1}},B=A_{i_{2}},C=A_{i_{3}}$. Let us prove that $\{A,B,C\}$ is linearly dependent. Since the submatrix of $J(x)$ consisting of columns $i_{1},i_{2}$ and $i_{3}$ has rank at most $2$, we have that for all $x\in{\mathbb{R}}^{n}$, there is $(\alpha_{x},\beta_{x},\gamma_{x})\neq(0,0,0)$ such that $\alpha_{x}Ax+\beta_{x}Bx+\gamma_{x}Cx=0$ and $\alpha_{x}+\beta_{x}+\gamma_{x}=0$. That is,

[TABLE]

Let $v_{1},\dots,v_{n}\in{\mathbb{R}}^{n}$ be an orthonormal basis of eigenvectors of the symmetric matrix $B-C$ with corresponding eigenvalues $\lambda_{1},\dots,\lambda_{n}\in{\mathbb{R}}$. If all eigenvalues are zero, then $B=C$ and the result follows. Suppose $\lambda_{1}\neq 0$ and let us apply (4.2) with $x=v_{1}$:

[TABLE]

For $i=1,\dots,n$ let us take the inner product with $v_{i}$. We have:

[TABLE]

Since $(\alpha_{1},\beta_{1})\neq(0,0)$ and $\lambda_{1}\neq 0$, we have $\alpha_{1}\neq 0$. That is,

[TABLE]

Now, for each $i=2,\dots,n$, let us repeat the construction for $x=v_{1}+v_{i}$. From (4.2), there is some $(\alpha_{i},\beta_{i})\neq(0,0)$ such that:

[TABLE]

Taking the inner product with $v_{1}$ and using (4.3) and the fact that $v_{1}^{\mathtt{T}}(A-C)v_{i}=v_{i}^{\mathtt{T}}(A-C)v_{1}$, we have:

[TABLE]

which implies ${\alpha}_{i}\neq 0$ and $\frac{{\beta}_{i}}{{\alpha}_{i}}=\frac{\beta_{1}}{\alpha_{1}},i>1$. Thus, we conclude that for $u_{1}=v_{1},u_{2}=v_{1}+v_{2},\dots,u_{n}=v_{1}+v_{n}$ it holds

[TABLE]

where $\delta=\frac{\beta_{1}}{\alpha_{1}}$. Since $\{u_{1},\dots,u_{n}\}$ is a basis of ${\mathbb{R}}^{n}$, we have $A-C+\delta(B-C)=0$, and the result follows.

∎

Note that Lemma 3 does not hold without the assumption of symmetry of the matrices. A counter-example is A_{1}=\left(\begin{array}[]{cc}1&0\\ 0&0\end{array}\right), A_{2}=\left(\begin{array}[]{cc}1&0\\ 0&1\end{array}\right) and A_{3}=\left(\begin{array}[]{cc}1&0\\ 1&0\end{array}\right) with $a=1$. We give next our result for problem (4.1).

Theorem 3**.**

If $(x^{*},z^{*})=(0,0)$ is a local minimizer of (4.1) and the rank of the Jacobian matrix $J(x,z)=[\nabla_{x,z}g_{1}(x,z)\cdots\nabla_{x,z}g_{m}(x,z)]$ increases at most by one in a neighborhood with respect to the rank at $(x,z)=(x^{*},z^{*})$, then there exists $t\in\Lambda_{m}$ such that $\sum_{i=1}^{m}t_{i}A_{i}$ is positive semidefinite.

Proof.

We have \nabla_{x,z}g_{i}(x,z)=\left(\begin{array}[]{c}A_{i}x\\ -1\end{array}\right),i=1,\dots,m. Hence, Mangasarian-Fromovitz holds at $(x^{*},z^{*})$, the critical cone $C(x^{*},z^{*})$ is the subspace ${\mathbb{R}}^{n}\times\{0\}$, and the set of Lagrange multipliers is the simplex $\Lambda_{m}$. Hence, its vertices are given by $\mu^{i}=e_{i},i=1,\dots,m$ where $e_{1},\dots,e_{m}$ is the canonical basis of ${\mathbb{R}}^{m}$ and $\nabla_{x}^{2}L(x^{*},z^{*},\mu^{i})=A_{i},i=1,\dots,m$. Since the rank of $J(x^{*})=J(x^{*},z^{*})$ is one, the rank assumption implies that $J(x)$ has rank at most $2$ in some neighborhood of $x^{*}$. This implies that $J(x)$ has rank at most $2$ for all $x\in{\mathbb{R}}^{n}$, and the result follows from Lemma 3 and Theorem 2.

Note that the local minimality of $(0,0)$ implies that $\max_{i=1,\dots,m}\{x^{\mathtt{T}}A_{i}x\}\geq 0$ for all $x$ in a small neighborhood of the origin, which implies it holds for all $x\in{\mathbb{R}}^{n}$, hence the result would also follow from Lemma 3 and Lemma 2.

∎

5 Conclusions

Although no-gap necessary and sufficient second-order optimality conditions of Fritz-John type are well established in the optimization literature [7], these type of optimality condition has little relations to second-order global convergence results of algorithms. Hence the importance of developing weak conditions ensuring the existence of a Lagrange multiplier with a positive semidefinite Lagrangian Hessian on the critical cone, or a meaningful subset.

We developed a second-order optimality condition of this type where the Lagrange multiplier set can be larger than a bounded line segment, generalizing [5]. This was done by an extension of Yuan’s Lemma [23] to a set of linearly dependent matrices. The results were applied to address a conjecture from [1] about a second-order optimality condition of this type under non-constant rank on a class of quadratically-constrained problems.

Acknowledgement

This work was supported by FAPESP (grants 2013/05475-7 and 2016/02092-8) and CNPq.