On a conjecture in second-order optimality conditions

TL;DR

This paper proves a conjecture on second-order optimality conditions in nonlinear programming, showing that under certain smoothness assumptions, a single Lagrange multiplier-based condition holds near local minimizers.

Contribution

The paper establishes the conjecture under the weaker assumption of a smooth singular value decomposition of the Jacobian, extending previous results.

Findings

Proves the conjecture under new smoothness assumptions.

Extends second-order optimality conditions to broader cases.

Provides a review of related literature.

Abstract

In this paper we deal with optimality conditions that can be verified by a nonlinear optimization algorithm, where only a single Lagrange multiplier is avaliable. In particular, we deal with a conjecture formulated in [R. Andreani, J.M. Martinez, M.L. Schuverdt, "On second-order optimality conditions for nonlinear programming", Optimization, 56:529--542, 2007], which states that whenever a local minimizer of a nonlinear optimization problem fulfills the Mangasarian-Fromovitz Constraint Qualification and the rank of the set of gradients of active constraints increases at most by one in a neighborhood of the minimizer, a second-order optimality condition that depends on one single Lagrange multiplier is satisfied. This conjecture generalizes previous results under a constant rank assumption or under a rank deficiency of at most one. In this paper we prove the conjecture under the additional assumption that the Jacobian matrix has a smooth singular value decomposition, which is weaker than previously considered assumptions. We also review previous literature related to the conjecture.

Figures

Equations

$$\begin{array}[]{lll}\mbox{minimize }&f(x),\\ \mbox{subject to }&h_{i}(x)=0\ \ \forall i\in\mathcal{E}:=\{1,\dots,m\},\\ &g_{j}(x)\leq 0\ \ \forall j\in\mathcal{I}:=\{1,\dots,p\},\\ \end{array}$$

$$\nabla f(x^{\ast })+\sum _{i=1}^m{\lambda }_i\nabla h_i(x^{\ast })+\sum _{j=1}^p{\mu }_j\nabla g_j(x^{\ast })=0\text{ with }{\mu }_j{g}_j(x^{\ast })=0,\forall j,$$

$$d^{\mathtt{T}}({\nabla }^{2}f(x^{\ast })+\sum _{i=1}^m{\lambda }_i{\nabla }^2{h}_i(x^{\ast })+\sum _{j=1}^p{\mu }_j{\nabla }^2{g}_j(x^{\ast }))d\ge 0.$$

$$\nabla L_x^g(x^{\ast },{\lambda }_0,\lambda ,\mu )=0\text{ with }{\mu }^{\mathtt{T}}g(x^{\ast })=0,{\lambda }_0\ge 0,\mu \ge 0,({\lambda }_0,\lambda ,\mu )\ne (0,0,0).$$

$$C(x^{\ast }):=\{d\in {\mathbb{R}}^n\mid \nabla f(x^{\ast }{)}^{\mathtt{T}}d=0;\nabla h_i(x^{\ast }{)}^{\mathtt{T}}d=0,i\in \mathcal{E};\nabla g_j(x^{\ast }{)}^{\mathtt{T}}d\le 0,j\in A(x^{\ast })\}.$$

$$\left\{d\in\mathbb{R}^{n}:\begin{array}[]{lll}&\nabla h_{i}(x^{*})^{\mathtt{T}}d=0,\text{ for }i\in\mathcal{E},\nabla g_{j}(x^{*})^{\mathtt{T}}d=0,\text{ if }\mu_{j}>0\\ &\nabla g_{j}(x^{*})^{\mathtt{T}}d\leq 0,\text{ if }\mu_{j}=0,j\in A(x^{*})\end{array}\right\},$$

$$S(x^{\ast }):=\{d\in {\mathbb{R}}^n\mid \nabla h_i(x^{\ast }{)}^{\mathtt{T}}d=0,i\in \mathcal{E};\nabla g_j(x^{\ast }{)}^{\mathtt{T}}d=0,j\in A(x^{\ast })\}.$$

$$\text{mbox}foreachd\in C(x^{\ast }),\text{mbox}thereisamultiplier(\lambda ,\mu )\in \Lambda (x^{\ast })\text{mbox}suchthat{d}^{\mathtt{T}}{\nabla }_{xx}^{2}L(x^{\ast },\lambda ,\mu )d\ge 0.$$

$$\forall d\in C(x^{\ast }),\text{sup}\{d^{\mathtt{T}}{\nabla }_{xx}^{2}L(x^{\ast },\lambda ,\mu )d:(\lambda ,\mu )\in \Lambda (x^{\ast }))\}\ge 0.$$

$$d^{\mathtt{T}}{\nabla }_{xx}^2{L}^g(x^{\ast },{\lambda }_0,\lambda ,\mu )d\ge 0.$$

$$\forall d\in K,d^{\mathtt{T}}{\nabla }_{xx}^{2}L(x^{\ast },{\lambda }^K,{\mu }^K)d\ge 0.$$

$$\begin{array}[]{ll}\mbox{Minimize }&x_{3},\\ &x_{3}\geq 2\sqrt{3}x_{1}x_{2}-2x_{2}^{2},\\ &x_{3}\geq x_{2}^{2}-3x_{1}^{2},\\ &x_{3}\geq-2\sqrt{3}x_{1}x_{2}-2x_{2}^{2}.\\ \end{array}$$

$$\begin{array}[]{ll}\mbox{Minimize }&0,\\ \mbox{subject to }&x_{1}\leq 0,\\ &x_{1}+x_{2}x_{1}^{2}+x_{2}^{3}/3\leq 0,\\ &2x_{1}+x_{2}x_{1}^{2}+x_{2}^{3}/3\leq 0,\end{array}$$

$$\begin{array}[]{lll}\mbox{minimize }&-x_{1}^{2}-x_{2},\\ \mbox{subject to }&2x_{1}^{2}+x_{2}\leq 0,\\ &-x_{1}^{2}+x_{2}\leq 0,\\ &x_{2}\leq 0.\end{array}$$

$$x\mapsto U(x)\in {\mathbb{R}}^{(m+q)\times (m+q)},x\mapsto \Sigma (x)\in {\mathbb{R}}^{(m+q)\times n}\text{mbox}andx\mapsto V(x)\in {\mathbb{R}}^{n\times n},$$

$$\nabla g_i(x{)}^{\mathtt{T}}=\sum _{k=1}^{r+1}{\sigma }_k(x)[u_k(x){]}_i{v}_k(x{)}^{\mathtt{T}},i=1,\dots ,p,$$

$${\nabla }^2{g}_i(x)=\sum _{k=1}^{r+1}{\sigma }_k(x)[u_k(x){]}_i{J}_{v_k}(x)+\sum _{k=1}^{r+1}([u_k(x){]}_i\nabla {\sigma }_k(x)+{\sigma }_k(x)\nabla [u_k(x){]}_i)v_k(x{)}^{\mathtt{T}},i=1,\dots ,p,$$

$$\displaystyle\begin{array}[]{rl}d^{\mathtt{T}}\nabla^{2}_{xx}L(x^{*},\mu)d=&d^{\mathtt{T}}\nabla^{2}fd+d^{\mathtt{T}}\left[\sum_{i=1}^{p}\mu_{i}\sum_{k=1}^{r+1}([u_{k}]_{i}\nabla\sigma_{k}+\sigma_{k}\nabla[u_{k}]_{i})v_{k}^{\mathtt{T}}+\sigma_{k}[u_{k}]_{i}J_{v_{k}}\right]d\\ =&d^{\mathtt{T}}\nabla^{2}fd+\sum_{k=1}^{r+1}[(d^{\mathtt{T}}\nabla\sigma_{k})(\mu^{\mathtt{T}}u_{k})+\sigma_{k}d^{\mathtt{T}}J_{u_{k}}^{\mathtt{T}}\mu]v_{k}^{\mathtt{T}}d+\sum_{k=1}^{r+1}\sigma_{k}(\mu^{\mathtt{T}}u_{k})d^{\mathtt{T}}J_{v_{k}}d.\end{array}$$

$$d^{\mathtt{T}}{\nabla }_{xx}^{2}L(x^{\ast },\mu )d=d^{\mathtt{T}}{\nabla }^{2}fd+(d^{\mathtt{T}}\nabla {\sigma }_{r+1})({\mu }^{\mathtt{T}}{u}_{r+1})(v_{r+1}^{\mathtt{T}}d)+\sum _{k=1}^r{\sigma }_k({\mu }^{\mathtt{T}}{u}_k)d^{\mathtt{T}}{J}_{v_k}d.$$

$$d^{\mathtt{T}}{\nabla }_{xx}^{2}L(x^{\ast },\mu )d=d^{\mathtt{T}}{\nabla }^{2}fd+(d^{\mathtt{T}}\nabla {\sigma }_{r+1})({\bar{\mu }}^{\mathtt{T}}{u}_{r+1}+t_{r+1})(v_{r+1}^{\mathtt{T}}d)+\sum _{k=1}^r{\sigma }_k({\bar{\mu }}^{\mathtt{T}}{u}_k)d^{\mathtt{T}}{J}_{v_k}d.$$

$$\underset{\mu \in \Lambda (x^{\ast })}{\max }{d}^{\mathtt{T}}{\nabla }_{xx}^{2}L(x^{\ast },\mu )d\ge 0,\forall d\in S(x^{\ast }).$$

$$M:=\{(t_{r+1},t_{r+2},\dots ,t_p)\in {\mathbb{R}}^{p-r}\text{ such that }\bar{\mu }+\sum _{j=r+1}^p{t}_j{u}_j\in \Lambda (x^{\ast })\},$$

$$a_{\ast }:=\text{ Inf }\{t_{r+1}:(t_{r+1},t_{r+2},\dots ,t_p)\in M\},$$

$$b_{\ast }:=\text{ Sup }\{t_{r+1}:(t_{r+1},t_{r+2},\dots ,t_p)\in M\}.$$

$$\max \{\theta (t_{r+1},d):t_{r+1}\in [a_{\ast },b_{\ast }]\}.$$

$$\max \{d^{\mathtt{T}}Pd,d^{\mathtt{T}}Qd\}\ge 0,\forall d\in S(x^{\ast }).$$

$$d^{\mathtt{T}}(\alpha P+\beta Q)d\ge 0,\forall d\in S(x^{\ast }).$$

$$\begin{array}[]{ll}\mbox{Minimize }&x_{3},\\ \mbox{subject to }&\cos(x_{1}+x_{2})-x_{3}-1\leq 0,\\ &-\cos(x_{1}+x_{2})-x_{3}+1\leq 0,\\ &-2x_{3}\leq 0,\end{array}$$

$$J(x)=\left(\begin{array}[]{ccc}-\sin(x_{1}+x_{2})&-\sin(x_{1}+x_{2})&-1\\ \sin(x_{1}+x_{2})&\sin(x_{1}+x_{2})&-1\\ 0&0&-2\end{array}\right),$$

$$\displaystyle U(x):=\left(\begin{array}[]{ccc}-\frac{1}{\sqrt{6}}&-\frac{1}{\sqrt{2}}&\frac{1}{\sqrt{3}}\\ -\frac{1}{\sqrt{6}}&\frac{1}{\sqrt{2}}&\frac{1}{\sqrt{3}}\\ -\frac{2}{\sqrt{6}}&0&-\frac{1}{\sqrt{3}}\end{array}\right),\Sigma(x):=\left(\begin{array}[]{ccc}\sqrt{6}&0&0\\ 0&2\sin(x_{1}+x_{2})&0\\ 0&0&0\end{array}\right)\mbox{ and}$$

Full text

Keywords: Nonlinear optimization, Constraint qualifications, Second-order optimality conditions, Singular value decomposition.

AMS Classification: 90C46, 90C30

1 Introduction

This paper considers a conjecture about second-order necessary optimality conditions for constrained optimization. Our interest in such conjecture comes from practical considerations. Numerical optimization deals with the design of algorithms with the aim of finding a point with the lowest possible value of a certain function over a constraint set. Useful tools for the design of algorithms are the necessary optimality conditions, i.e., conditions satisfied by every local minimizer. Not all necessary optimality conditions serve that purpose. Optimality conditions must be computable with the information provided by the algorithm, where its fulfillment indicates that the considered point is an acceptable solution. For constrained optimization problems, the Karush-Kuhn-Tucker (KKT) conditions are the basis for most optimality conditions. In fact, most algorithms for constrained optimization are iterative and in their implementation, the KKT conditions serve as a theoretical guide for developing suitable stopping criteria. For more details, see [49, Framework 7.13, page 513], [33, Chapter 12] and [9].

Necessary optimality conditions can be of first- or second-order depending on whether the first- or second-order derivatives are used in the formulation. When the second-order information is avaliable, one can formulate second-order conditions. Such conditions are much stronger than first-order ones and hence are mostly desirable, since they allow us to rule out possible non-minimizers accepted as solution when we only use first-order information.

Global convergence proofs of second-order algorithms are based on second-order necessary optimality condition of the form: If a local minimizer satisfies some constraint qualification, then the WSOC condition holds, where WSOC stands for the Weak Second-order Optimality Condition, that states that the Hessian of the Lagrangian at a KKT point, for some Lagrange multiplier, is positive semidefinite on the subspace orthogonal to the gradients of active constraints, see Definition 2.1.

Thus, we are interested in assumptions guaranteeing that local minimizers satisfy WSOC, given its implications to numerical algorithms. The conjecture comes along these lines. In order to precisely state the conjecture, we need some definitions.

Consider the nonlinear constrained optimization problem

[TABLE]

where $f$, $h_{i}$, $g_{j}:\mathbb{R}^{n}\rightarrow\mathbb{R}$ are assumed to be, at least, twice continuously differentiable functions.

Denote by $\Omega$ the feasible set of (1.1). For a point $x\in\Omega$, we define $A(x):=\{j\in\mathcal{I}:g_{j}(x)=0\}$ to denote the set of indices of active inequalities. A feasible point $x^{*}$ satisfies the Mangasarian-Fromovitz Constraint Qualification (MFCQ) if $\{\nabla h_{i}(x^{*}):i\in\mathcal{E}\}$ is a linearly independent set and there is a direction $d\in{\mathbb{R}}^{n}$ such that $\nabla h_{i}(x^{*})^{\mathtt{T}}d=0,i\in\mathcal{E}$ and $\nabla g_{j}(x^{*})^{\mathtt{T}}d<0$, $j\in A(x^{*})$. Define by $J(x)$ the matrix whose first $m$ rows are formed by $\nabla h_{i}(x)^{\mathtt{T}}$, $i\in\mathcal{E}$ and the remaining rows by $\nabla g_{j}(x)^{\mathtt{T}},j\in A(x^{*})$.

In [7], with the aim of stating a verifiable condition guaranteeing global convergence of a second-order augmented Lagrangian method to a second-order stationary point, the authors proposed a new condition [7, Section 3] suitable for that purpose. Furthermore, based on [14] and their recently proposed condition, they stated the following conjecture, see [7, Section 5]:

Conjecture**.**

Let $x^{*}$ be a local minimizer of (1.1). Assume that:

MFCQ holds at $x^{*}$, 2. 2.

the rank of $\{\nabla h_{i}(x),\nabla g_{j}(x):i\in\mathcal{E};j\in A(x^{*})\}$ is at most $r+1$ in a neighborhood of $x^{*}$, where $r$ is the rank of $\{\nabla h_{i}(x^{*}),\nabla g_{j}(x^{*}):i\in\mathcal{E};j\in A(x^{*})\}$.

Then, there exists a Lagrange multiplier $(\lambda,\mu)\in{\mathbb{R}}^{m}\times{\mathbb{R}}^{p}_{+}$ such that

[TABLE]

and for every $d\in\mathbb{R}^{n}$ such that $\nabla h_{i}(x^{*})^{\mathtt{T}}d=0$, $\forall i$; $\nabla g_{j}(x^{*})^{\mathtt{T}}d=0$, $\forall j\in A(x^{*})$, we have

[TABLE]

Note that (1.2)-(1.3) is the WSOC condition. We are aware of two previous attempts of solving this conjecture. A proof of it under an additional technical condition has appeared, recently, in [53]. Also, a counter-example appeared in [44]. As we will see later in Section 3, these results are incorrect. Also, the recent paper [39] proved the conjecture for a special form of quadratically-constrained problems. Our approach is different from the ones mentioned above and it is based on an additional assumption of smoothness of the singular value decomposition of $J(x)$ around the basis point $x^{*}$.

As we have mentioned, WSOC has two important features that makes the Conjecture relevant in practical algorithms, which is our main motivation for pursuing it. The first one is that it does not rely on the whole set of Lagrange multipliers, in contrast with other second-order conditions in the literature, and the second one is that positive semi-definiteness of the Hessian of the Lagrangian must be verified in a subspace (a more tractable task) rather than at a pointed cone.

This is compatible with the implementation of an algorithm that globally converges to a point $x^{*}$ fulfilling WSOC. At each iteration, one has available an aproximation $x^{k}$ to a solution and a single Lagrange multiplier approximation $(\lambda^{k},\mu^{k})$ and one may check if WSOC is approximately satisfied at the current point $(x^{k},\lambda^{k},\mu^{k})$ if one wishes to declare convergence to a second-order stationary point (see details in [4] and references therein). Of course, this is still a non-trivial computational task, so this only makes sense when most of the effort to check WSOC was already done as part of the computation of the iterate. This is the case of algorithms that try to compute a descent direction and a negative curvature direction [2, 47]. Near a KKT point, once the procedure for computing the negative curvature direction fails, WSOC is approximately satisfied.

This is an important difference with respect to other conditions that we review in the next section. In order to verify an optimality condition that relies on the whole set of Lagrange multipliers, one needs an algorithm that generates all multipliers, which may be difficult. Even more, in classical second-order conditions, one must check if a matrix is positive semi-definite on a pointed cone, which is a far more difficult problem than checking it on a subspace (see [48]). Finally, we are not aware of any reasonable iterative algorithm that generates subsequences that converges to a point that satisfies a classical, more accurate, second-order optimality condition based on a pointed cone. The discussion in [37] indicates that such algorithms probably do not exist.

In Section 2 we briefly review some related results on second-order optimality conditions. In Section 3 we prove the Conjecture under the additional assumption that the singular value decomposition of $J(x)$ is smooth in a neighborhood of $x^{*}$.In Section 4 we present some conclusions and future directions of research on this topic.

2 Second-order optimality conditions

In this section, we review some classical and some recent results on second-order optimility conditions. Several second-order optimility conditions have been proposed in the literature, both from a theoretical and practical point of view, see [15, 52, 49, 33, 32, 17, 20, 51, 22, 19, 12, 14, 26, 34, 18] and references therein.

First, we start with the basic notation. $\mathbb{R}^{n}$ stands for the $n$-dimensional real Euclidean space, $n\in\mathbb{N}$. $\mathbb{R}_{+}^{n}\subset\mathbb{R}^{n}$ is the set of vector whose components are nonnegative. The canonical basis of ${\mathbb{R}}^{n}$ is denoted by $e_{1},\dots,e_{n}$. A set $\mathcal{R}\subset\mathbb{R}^{n}$ is a ray if $\mathcal{R}:=\{rd_{0}:r\geq 0\}$ for some $d_{0}\in\mathbb{R}^{n}$. Given a convex cone $\mathcal{K}\subset\mathbb{R}^{n}$, we define the lineality set of $\mathcal{K}$ as $\mathcal{K}\cap-\mathcal{K}$, which is the largest subspace contained in $\mathcal{K}$. We say that $\mathcal{K}$ is a first-order cone if $\mathcal{K}$ is the direct sum of a subspace and a ray.

We denote the Lagrangian function by $L(x,\lambda,\mu)=f(x)+\sum_{i=1}^{m}\lambda_{i}h_{i}(x)+\sum_{j=1}^{p}\mu_{j}g_{j}(x)$ where $(x,\lambda,\mu)$ is in $\mathbb{R}^{n}\times\mathbb{R}^{m}\times\mathbb{R}_{+}^{p}$ and the generalized Lagrangian function as $L^{g}(x,\lambda_{0},\lambda,\mu)=\lambda_{0}f(x)+\sum_{i=1}^{m}\lambda_{i}h_{i}(x)+\sum_{j=1}^{p}\mu_{j}g_{j}(x)$ where $(x,\lambda_{0},\lambda,\mu)\in\mathbb{R}^{n}\times\mathbb{R}_{+}\times\mathbb{R}^{m}\times\mathbb{R}_{+}^{p}$. Clearly, $L^{g}(x,1,\lambda,\mu)=L(x,\lambda,\mu)$. The symbols $\nabla_{x}L^{g}(x,\lambda_{0},\lambda,\mu)$ and $\nabla^{2}_{xx}L^{g}(x,\lambda_{0},\lambda,\mu)$ stand for the gradient and the Hessian of $L^{g}(x,\lambda_{0},\lambda,\mu)$ with respect to $x$, respectively. Similar notation holds for $L(x,\lambda,\mu)$.

The generalized first-order optimality condition at the feasible point $x^{*}$ is

[TABLE]

The set of vectors $(\lambda_{0},\lambda,\mu)\in\mathbb{R}_{+}\times\mathbb{R}^{m}\times\mathbb{R}_{+}^{p}$ satisfying (2.1) is the set of generalized Lagrange multipliers (or Fritz John multipliers), denoted by $\Lambda_{0}(x^{*})$. Note that (2.1) with $\lambda_{0}=1$ corresponds to the Karush-Kuhn-Tucker (KKT) conditions, the standard first-order condition in numerical optimization. We denote by $\Lambda(x^{*}):=\{(\lambda,\mu)\in\mathbb{R}^{m}\times\mathbb{R}_{+}^{p}:(1,\lambda,\mu)\in\Lambda_{0}(x^{*})\}$, the set of all Lagrange multipliers. At every minimizer, there are Fritz John multipliers such that (2.1) holds, that is, $\Lambda_{0}(x^{*})\neq\emptyset$. In order to get existence of true Lagrange multipliers, additional assumptions have to be required. Assumptions on the analytic description of the feasible set that guarantee the validity of the KKT conditions at local minimizers are called constraint qualification (CQ). Thus, under any CQ, the KKT conditions are necessary for optimality.

When the second-order information is avaliable, we can consider second-order conditions. In order to describe second-order conditions (in a dual form), we introduce some important sets. We start with the cone of critical directions (critical cone), defined as follows:

[TABLE]

Obviouly, $C(x^{*})$ is a non-empty closed convex cone. When $\Lambda(x^{*})\neq\emptyset$, the critical cone $C(x^{*})$ can be written as

[TABLE]

for every $(\lambda,\mu)\in\Lambda(x^{*})$. From the algorithmic point of view, an important set is the critical subspace (or weak critical cone), given by:

[TABLE]

In the case when $\Lambda(x^{*})\neq\emptyset$, a simple inspection shows that the critical subspace $S(x^{*})$ is the lineality space of the critical cone $C(x^{*})$. Under strict complementarity, $S(x^{*})$ coincides with $C(x^{*})$.

Now, we are able to define the classical second-order conditions.

Definition 2.1**.**

Let $x^{*}$ be a feasible point with $\Lambda(x^{*})\neq\emptyset$. We have the following definitions

We say that the strong second-order optimality condition (SSOC) holds at $x^{*}$ if there is a $(\lambda,\mu)\in\Lambda(x^{*})$ such that $d^{\mathtt{T}}\nabla^{2}_{xx}L(x^{*},\lambda,\mu)d\geq 0$ for every $d\in C(x^{*})$. 2. 2.

We say that the weak second-order optimality condition (WSOC) holds at $x^{*}$ if there is a $(\lambda,\mu)\in\Lambda(x^{*})$ such that $d^{\mathtt{T}}\nabla^{2}_{xx}L(x^{*},\lambda,\mu)d\geq 0$ for every $d\in S(x^{*})$.

The classical second-order condition SSOC is particularly important from the point of view of passing from necessary to sufficient optimality conditions. In this case, strenghtening the sign of the inequality from “$\geq 0$” to “$>0$” in the definition of SSOC, that is, instead of positive semi-definiteness of the Hessian of the Lagrangian on the critical cone, we require its positive definiteness on the same cone (minus the origin), we get a sufficient optimality condition, see [15, 20]. Furthermore, this sufficient condition also ensures that the local minimizer $x^{*}$ is isolated. Besides these nice properties, from the practical point of view, SSOC has some disvantages. In fact, to verify the validity of SSOC at a given point, is in general, an NP-hard problem, [48, 50]. Also, it is well known that very simple second-order methods fail to generate sequences in which SSOC holds at its accumulation points, see [37].

From this point of view, WSOC seems to be the most adequate second-order condition when dealing with global convergence of second-order methods. In fact, all second-order algorithms known by the authors only guarantee convergence to points satisfying WSOC, see [2, 21, 23, 24, 28, 27, 29, 31, 35, 47] and references therein.

This situation in which a most desirable theoretical property is not suitable in an algorithmic framework is not particular only to the second-order case. Even in the first-order case, it is known, for example, that the Guignard constraint qualification is the weakest possible assumption to yield KKT conditions at a local minimizer [36]. In other words, a good first-order necessary optimality condition is of the form “KKT or not-Guignard”. But this is too strong for practical purposes, since no algorithm is known to fulfill such condition at limit points of sequences generated by it, in fact, the convergence assumptions of algorithms require stronger constraint qualifications [5, 6, 8, 9]. For second-order algorithms, the situation is quite similar, with the peculiarity that the difficulty is not only on the required constraint qualification, but also in the verification of the optimality condition, since, numerically, we can only guarantee a partial second-order property, that is, for directions in the critical subspace, which is a subset of the desirable critical cone of directions.

As the KKT conditions, SSOC and WSOC hold at minimizers only if some additional condition is valid. As we will explore in the next section, only MFCQ is not enough to ensure the existence of some Lagrange multiplier where SSOC holds. Even WSOC can not be assured to hold under MFCQ alone. Under MFCQ, we have the following result, [20, 16]:

Theorem 2.1**.**

Let $x^{*}$ be a local minimizer of (1.1). Assume that MFCQ holds at $x^{*}$. Then,

[TABLE]

Note that for each critical direction, we have an associated Lagrange multiplier $(\lambda,\mu)\in\Lambda(x^{*})$, in opposition to SSOC or WSOC, where we require the same Lagrange multiplier for all critical directions. Observe that (2.5) does not imply WSOC (and neither SSOC).

Observe also that since $\Lambda(x^{*})$ is a compact set (by MFCQ), (2.5) can be written in a more compact form, namely,

[TABLE]

Although this optimality condition relies on the whole Lagrange multiplier set $\Lambda(x^{*})$, hence it is not suitable for our practical considerations, it will play a crucial role in our analysis.

Even when no constraint qualification is assumed, a second-order optimality condition can be formulated, relying on Fritz John multipliers (2.1):

Theorem 2.2**.**

Let $x^{*}$ be a local minimizer of (1.1). Then, for every $d$ in the critical cone $C(x^{*})$, there is a Fritz John multiplier $(\lambda_{0},\lambda,\mu)\in\Lambda_{0}(x^{*})$ such that

[TABLE]

The optimality condition of Theorem 2.2 has been studied a lot over the years, [30, 16, 42, 20, 11]. An important property is that it can be transformed into a sufficient optimality condition by simply replacing the non-negative sign “$\geq 0$” by “$>0$” (except at the origin), without any additional assumption. For this reason, this condition is said to be a “no-gap” optimality condition. Note that this is different from the case of SSOC, since an additional assumption must be made for the necessary condition to hold. Note that Theorem 2.1 can be derived from Theorem 2.2, since under MFCQ, there is no Fritz John multiplier with $\lambda_{0}=0$.

We emphasize that even though optimality conditions given by Theorems 2.1 and 2.2 have nice theoretical properties, they do not suit our framework since their verification requires the knowledge of the whole set of (generalized) Lagrange multipliers at the basis point, whereas in practice, we only have access to (an approximation of) a single Lagrange multiplier. In the case of the optimality condition given by Theorem 2.2, one could argue that the possibility of verifying it with $\lambda_{0}=0$, and hence independently of the objective function, is not useful at all as an optimality condition. This is arguably the case for the first-order Fritz John optimality condition, but since Theorem 2.2 gives a “no-gap” optimality condition, this argument is not convincent in the second-order case. In fact, one could show that if the sufficient optimality condition associated to Theorem 2.2 is fulfilled with $\lambda_{0}=0$ for all critical directions, then the basis point is an isolated feasible point, and hence a local solution independently of the objective function. We take the point of view that algorithms naturally treat differently the objective function and the constraint functions, in a way that a multiplier associated to the objective function is not present, hence our focus on Lagrange multipliers, rather than on Fritz John multipliers.

As we have mentioned, known practical methods are only guaranteed to converge to points satisfying WSOC, and hence, we focus our attention, from now on, on conditions ensuring it at local minimizers.

We start with [14], where the authors investigate the issue of verifying (2.5) for the same Lagrange multiplier:

Theorem 2.3** ([14]).**

Let $x^{*}$ be a local minimizer of (1.1). Assume that MFCQ holds at $x^{*}$ and that $\Lambda(x^{*})$ is a line segment. Then, for every first-order cone $K\subset C(x^{*})$, there is a $(\lambda^{K},\mu^{K})\in\Lambda(x^{*})$ such that

[TABLE]

We are interested only in the special case $K:=S(x^{*})$. Thus, (2.7) holds at a local minimizer $x^{*}$ when $\Lambda(x^{*})$ is a line segment and MFCQ holds at $x^{*}$ (or, equivalently, $\Lambda(x^{*})$ is a bounded line segment). Note that in this case, (2.7) is equivalent to WSOC.

In order to prove Theorem 2.3 a crucial result is Yuan’s Lemma [55], which was generalized for first-order cones in [14]. For further applications of Yuan’s Lemma, see [43, 25].

Lemma 2.4** (Yuan [55, 14]).**

Let $P,Q\in{\mathbb{R}}^{n\times n}$ be two symmetric matrices and $K\subset{\mathbb{R}}^{n}$ a first-order cone. Then the following conditions are equivalent:

• •

$\max\{d^{\mathtt{T}}Pd,d^{\mathtt{T}}Qd\}\geq 0,\ \ \forall d\in K$;

• •

There exist $\alpha\geq 0$ and $\beta\geq 0$ with $\alpha+\beta=1$ such that $d^{\mathtt{T}}(\alpha P+\beta Q)d\geq 0$, $\forall d\in K$.

A sufficient condition to guarantee that $\Lambda(x^{*})$ is a line segment, is to require that the rank of the Jacobian matrix $J(x^{*})$ is row-deficient by at most one, that is, the rank is one less than the number of rows. The fact that the rank assumption yields the one-dimensionality of $\Lambda(x^{*})$ is a simple consequence of the rank-nullity theorem. Thus, we have the following result:

Theorem 2.5** (Baccari and Trad [14]).**

Let $x^{*}$ be a local minimizer of (1.1) such that MFCQ holds and the rank of the Jacobian matrix $J(x^{*})\in{\mathbb{R}}^{(m+q)\times n}$ is $m+q-1$, where $q$ is the number of active inequality constraints at $x^{*}$. Then, there exists a Lagrange multiplier $(\lambda,\mu)\in{\mathbb{R}}^{m}\times{\mathbb{R}}^{p}_{+}$ such that WSOC holds.

Another line of reasoning in order to arrive at second-order optimality conditions is to use Janin’s version of the classical Constant Rank theorem ([54], Theorem 2.9). See [41, 3, 46].

Theorem 2.6** (Constant Rank).**

Let $x^{*}\in\Omega$ and $d\in C(x^{*})$. Let $E\subset\{1,\dots,p\}$ be the set of indices $j$ such that $\nabla g_{j}(x^{*})^{\mathtt{T}}d=0,j\in A(x^{*})$. If $\{\nabla h_{i}(x),i\in\mathcal{E};\nabla g_{j}(x),j\in E\}$ has constant rank in a neighborhood of $x^{*}$, then, there are $\varepsilon>0$ and a twice continuously differentiable function $\xi:(-\varepsilon,\varepsilon)\to{\mathbb{R}}^{n}$ such that $\xi(0)=x^{*},\xi^{\prime}(0)=d,h_{i}(\xi(t))=0,i\in\mathcal{E};g_{j}(\xi(t))=0,j\in E$ for $t\in(-\varepsilon,\varepsilon)$ and $g(\xi(t))\leq 0$ for $t\in[0,\varepsilon)$.

The proof that the function $\xi$ is twice continuously differentiable was done in [46].

Using a constant rank assumption jointly with MFCQ, in [7], Andreani, Martínez and Schuverdt have proved the existence of multipliers satisfying WSOC at a local minimizer as stated below. This joint condition was also used in the convergence analysis of a second-order augmented Lagrangian method.

Theorem 2.7** (Andreani, Martínez and Schuverdt [7]).**

Let $x^{*}$ be a local minimizer of (1.1) with MFCQ holding at $x^{*}$. Assume that the rank of the Jacobian matrix $J(x)\in{\mathbb{R}}^{(m+q)\times n}$ is constant around $x^{*}$, where $q$ is the number of active inequality constraints at $x^{*}$. Then, WSOC holds at $x^{*}$.

The proof can be done using Theorem 2.6 for $d\in S(x^{*})$ and $E=\{1,\dots,p\}$, using the fact that $t=0$ is a local minimizer of $f(\xi(t)),t\geq 0$.

This result was further improved in [3], where they noticed that MFCQ can be replaced by the non-emptyness of $\Lambda(x^{*})$. This was also done independently in [38]. In fact, WSOC can be proved to hold for all Lagrange multipliers:

Theorem 2.8** (Andreani, Echagüe and Schuverdt [3]).**

Let $x^{*}$ be a local minimizer of (1.1) such that the rank of the Jacobian matrix $J(x)\in{\mathbb{R}}^{(m+q)\times n}$ is constant around $x^{*}$, where $q$ is the number of active inequality constraints at $x^{*}$. Then, every Lagrange multiplier $(\lambda,\mu)\in{\mathbb{R}}^{m}\times{\mathbb{R}}^{p}_{+}$ (if any exists) is such that WSOC holds.

This same technique can be employed under the Relaxed Constant Rank CQ (RCRCQ, [45]), that is, $\{\nabla h_{i}(x),i\in\mathcal{E};\nabla g_{j}(x),j\in E\}\mbox{ has constant rank around }x^{*}\mbox{ for every }E\subset A(x^{*}),$ to prove the stronger result that all Lagrange multipliers satisfy SSOC. See [3, 46]. These results can be strengthened by replacing the use of the Constant Rank theorem by the assumption that the critical cone is a subset of the Tangent cone of a modified feasible set (Abadie-type assumptions). See details in [1, 18].

3 The conjecture

In this section we prove the conjecture under an additional assumption based on the smoothness of the singular value decomposition of the Jacobian matrix. In view of Theorems 2.5 and 2.7, that arrives at the same second-order optimality condition under MFCQ and row-rank deficiency of at most one or under MFCQ and the constant rank assumption of the Jacobian $J(x)$, it is natural to conjecture that the same result would hold under MFCQ and assuming that the rank increases at most by one in a neighborhood of a local minimizer. This was conjectured in [7]. Although an unification of both results would be interesting, this was a bold conjecture since the theorems have completely different proofs.

Let us first show that Baccari and Trad’s result can be generalized in order to consider column-rank deficiency. The proof is a simple application of the rank-nullity theorem.

Theorem 3.1**.**

Let $x^{*}$ be a local minimizer of (1.1) such that MFCQ holds and the rank of the Jacobian matrix $J(x^{*})\in{\mathbb{R}}^{(m+q)\times n}$ is $n-1$, where $q$ is the number of active inequality constraints at $x^{*}$. Then, there exists a Lagrange multiplier $(\lambda,\mu)\in{\mathbb{R}}^{m}\times{\mathbb{R}}^{p}_{+}$ such that WSOC holds.

Proof.

Applying the rank-nullity theorem to $S(x^{*})=\text{Ker}(J(x^{*}))$, we get that $\mbox{dim}(S(x^{*}))=1$. Hence, there is $d_{0}\in S(x^{*})$ such that $S(x^{*})=\{td_{0},t\in{\mathbb{R}}\}$. Since MFCQ holds, Theorem 2.1 yields (2.5). In particular, for $d=d_{0}$, there is a Lagrange multiplier $(\lambda,\mu)$ such that $d_{0}^{\mathtt{T}}\nabla^{2}_{xx}L(x^{*},\lambda,\mu)d_{0}\geq 0$. This same Lagrange multiplier can be used for all other directions $d=td_{0}\in S(x^{*})$, since $d^{\mathtt{T}}\nabla^{2}_{xx}L(x^{*},\lambda,\mu)d=t^{2}d_{0}^{\mathtt{T}}\nabla^{2}_{xx}L(x^{*},\lambda,\mu)d_{0}\geq 0$. Thus, WSOC holds at $x^{*}$. ∎

The previous results show that the Conjecture is true in dimension less than or equal to two, or when there are at most two active constraints. In ${\mathbb{R}}^{3}$, the remarkable example by Arutyunov [11]/Anitescu [10] shows that if the rank increases by more than two around $x^{*}$, WSOC may fail for all Lagrange multipliers (also, SSOC fails).

We describe below a modification of the example, given in [13], since it gives nice insights about the Conjecture.

Example 3.1**.**

[TABLE]

Here $x^{*}=(0,0,0)$ is a global minimizer. The critical subspace is the whole plane $x_{3}=0$ and $\Lambda(x^{*})$ is the simplex $\mu_{1}+\mu_{2}+\mu_{3}=1,\mu_{j}\geq 0,j=1,2,3$. Figure 1 shows the graph of the right-hand side of each constraint, where the feasible set is the set of points above all surfaces. Note that along every direction in the critical cone, there is a convex combination of the constraints that moves upwards and (2.5) holds, but for any convex combinations of the constraints, there exists a direction in the critical cone that moves downwards. This means that WSOC fails for all Lagrange multipliers. Since in this example $C(x^{*})=S(x^{*})$, this means that SSOC also fails for all Lagrange multipliers. Note also in Figure 1 that around $x^{*}$ there is no feasible curve such that all constraints are active along this curve, which is the main property allowing the proof of WSOC under constant rank assumptions (Theorem 2.6).

Before describing our proof, we briefly point out previous attempts of solving the Conjecture. In [53], the authors stated the validity of the Conjecture with the additional following assumption:

Assumption (A3) [53]:

If there exists a sequence $\{x^{k}\}$ converging to $x^{*}$ such that the rank of $J(x^{k})$ is $r+1$ for all $k$, then for any $x^{k}$ and any subset $E\subset A(x^{*})$, the rank of $\{\nabla h_{i}(x),i\in\mathcal{E};\nabla g_{j}(x),j\in E\}$ is constant around $x^{k}$.

The proof of the Conjecture under (A3) in [53] is based on the following incorrect Lemma:

Lemma 3.4 from [53]: Under (A3) and the assumptions of the Conjecture, whenever there exists a sequence $\{x^{k}\}$ converging to $x^{*}$ such that the rank of $J(x^{k})$ is $r+1$ for all $k$, there exists some index set $E\subset A(x^{*})$ such that the rank of $\{\nabla g_{j}(x^{k}),j\in E\}$ is equal to $1$ and the rank of $\{\nabla h_{i}(x),i\in\mathcal{E};\nabla g_{j}(x),j\in A(x^{*})\backslash E\}$ is $r$ for an infinite number of indices $k$.

The following counter-example shows that it is incorrect.

Counter-Example 3.1**.**

[TABLE]

at $x^{*}=(0,0)$.

Clearly, $x^{*}=(0,0)$ is a local minimizer that fulfills MFCQ, the rank of the Jacobian is $1$ at $x^{*}$ and at most $2$ at every other point. Also, for any subset of $A(x^{*})=\{1,2,3\}$, the rank of the associated gradients is constant in the neighborhood of every point $x$ different from $x^{*}$, hence all assumptions of Lemma 3.4 from [53] are fulfilled. A simple inspection shows that for every point $x$ different from $x^{*}$ one can not separate the three gradients into two subsets of rank $1$ as the lemma states. In fact, every subset with two gradients will have rank $2$.

Another attempt to solve the Conjecture is given in [44]. Here, the following problem is presented as a counter-example for the Conjecture:

[TABLE]

The point $x^{*}:=(0,0)$ is a local minimizer that satisfies MFCQ, the rank of $J(x^{*})$ is $1$ and increases at most to $2$ around $x^{*}$. In [44], the authors only show that WSOC does not need to hold for all Lagrange multipliers. In particular, it was shown that it does not hold at $\bar{\mu}:=(0,1,0)$. Clearly, it does not disprove the Conjecture since there are other Lagrange multipliers that fulfill WSOC, as implied by Theorem 3.1. For instance, $\mu:=(1,0,0)$ satisfies WSOC.

Finally, we present our main result. We prove the Conjecture under an additional technical assumption on the smoothness of the singular value decomposition (SVD) of the Jacobian matrix around $x^{*}$.

Assumption 3.1**.**

Let $q$ be the number of active inequality constraints at $x^{*}$ and $J(x)\in{\mathbb{R}}^{(m+q)\times n}$ be the Jacobian matrix for $x$ near $x^{*}$. We assume that there exist differentiable functions around $x^{*}$ given by

[TABLE]

such that $J(x)=U(x)\Sigma(x)V(x)^{\mathtt{T}}$, where $\Sigma(x)$ is diagonal with diagonal elements $\sigma_{1}(x),\sigma_{2}(x),\dots,\sigma_{k}(x)$, where $k:=\min\{m+q,n\}$ and $\sigma_{i}(x)=0$ when $i$ is greater than the rank of $J(x)$. We assume also that $U(x^{*})$ and $V(x^{*})$ are matrices with non-zero orthogonal columns.

Note that only at $x=x^{*}$ we assume that $U(x^{*})$ and $V(x^{*})$ are matrices with orthogonal columns. This implies that $U(x)$ and $V(x)$ are at least invertible matrices in a small enough neighborhood of $x^{*}$, but not necessarily with orthogonal columns.

Note that we do not require that the columns of the matrices $U(x^{*})$ and $V(x^{*})$ to be normalized, as in the classical SVD decomposition. In our proof, it is also not necessary to adopt the convention of non-negativeness of $\sigma_{i}(x)$ or that they are ordered. Without risk of confusion, we will still call this weaker decomposition as the SVD. We introduce this extra freedom in the decomposition in order to allow more easily for the differentiability of the functions.

Theorem 3.2**.**

Assume that $x^{*}$ is a local minimizer of (1.1) that fulfills MFCQ. Let $r$ be the rank of $J(x^{*})$, and assume that for every $x$ in some neighborhood of $x^{*}$ the rank of $J(x)$ is at most $r+1$. Suppose also that Assumption 3.1 holds. Then, there is a Lagrange multiplier ($\lambda,\mu)$ such that WSOC holds.

Proof.

Let us consider the column functions $U(x)=[u_{1}(x)\dots u_{m+q}(x)]$ and $V(x)=[v_{1}(x)\dots v_{n}(x)]$. Clearly, $J(x)=\sum_{k=1}^{r+1}\sigma_{k}(x)u_{k}(x)v_{k}(x)^{\mathtt{T}}$. To simplify the notation, let us assume that $m=0$ and $A(x^{*})=\{1,\dots,p\}$, that is, $q=p$, hence, it holds that

[TABLE]

where $[u]_{i}$ is the $i$-th coordinate of the vector $u$ of appropriate dimension.

Since the functions are smooth we can compute derivatives and get

[TABLE]

where $J_{v_{k}}(x)\in{\mathbb{R}}^{n\times n}$ is the Jacobian matrix of the function $v_{k}$ at $x$.

Now, let us fix a direction $d\in S(x^{*})$ and a Lagrange multiplier $\mu\in\Lambda(x^{*})$ (we identify a Lagrange multiplier $(\lambda,\mu)$ with $\mu$ since we are assuming $m=0$). Now, we proceed to evaluate $d^{\mathtt{T}}\nabla^{2}_{xx}L(x^{*},\mu)d$. We omit the dependency on $x^{*}$ for simplicity. Then,

[TABLE]

From $S(x^{*})=\text{Ker}(J(x^{*}))$, and from the SVD, $J(x)=U(x)\Sigma(x)V(x)^{\mathtt{T}}$, we can conclude that there are $s_{r+1},\dots,s_{n}\in{\mathbb{R}}$ such that $d=\sum_{j=r+1}^{n}s_{j}v_{j}$. Hence, from the orthogonality of $\{v_{1},\dots,v_{p}\}$, we get $v_{k}^{\mathtt{T}}d=0$, $k<r+1$. Furthermore, since $\sigma_{r+1}=0$ we obtain that

[TABLE]

For a fixed Lagrange multiplier $\bar{\mu}\in\Lambda(x^{*})$ (note that MFCQ ensures non-emptyness of $\Lambda(x^{*})$), we can write $\Lambda(x^{*})=(\bar{\mu}+\text{Ker}(J(x^{*})^{\mathtt{T}}))\cap{\mathbb{R}}^{p}_{+}$, hence, there are $t_{r+1},\dots,t_{p}\in{\mathbb{R}}$ such that $\mu=\bar{\mu}+\sum_{j=r+1}^{p}t_{j}u_{j}$ and we can write (3.4) as

[TABLE]

Observe that for a fixed $d\in S(x^{*})$, the value of $d^{\mathtt{T}}\nabla^{2}_{xx}L(x^{*},\mu)d$, for $\mu\in\Lambda(x^{*})$, depends on a single parameter $t_{r+1}$. Since MFCQ holds, condition (2.5) holds, and we may write it as

[TABLE]

In virtue of the fact that the value of $d^{\mathtt{T}}\nabla^{2}_{xx}L(x^{*},\mu)d$ depends only on one parameter, in order to apply Yuan’s lemma, we will rewrite (3.6) as a maximization problem over a line segment. For that purpose, we define the set

[TABLE]

and the following optimization problems

[TABLE]

and

[TABLE]

Both values $a_{*}$ and $b_{*}$ are finite and attained since $M$ is a compact set. Furthermore, $M$ is a compact convex set. It is easy to see, that $M$ is convex and closed since $\Lambda(x^{*})$ is also convex and closed. To show that $M$ is bounded, suppose by contradiction, that there is a sequence $(t_{r+1}^{k},\dots,t_{p}^{k})\in M$ with $T_{k}:=\max\{|t^{k}_{r+1}|,\dots,|t_{p}^{k}|\}\rightarrow\infty$. Since $\Lambda(x^{*})$ is bounded, there is a scalar $K$ such that $\|\bar{\mu}+\sum_{j=r+1}^{p}t^{k}_{j}u_{j}\|\leq K$, $\forall k\in\mathbb{N}$. Dividing this expression by $T_{k}$ and taking an adequate subsequence, we conclude that there are $\bar{t}_{r+1},\dots,\bar{t}_{p}$ not all zero, such that $\sum_{j=r+1}^{p}\bar{t}_{j}u_{j}=0$, which is a contradiction with the linear independence of $\{u_{r+1},\dots,u_{p}\}$. Thus, $M$ must be a compact convex set.

Finally, denote by $\theta(t_{r+1},d):=d^{\mathtt{T}}\nabla^{2}fd+(\bar{\mu}^{\mathtt{T}}u_{r+1}+t_{r+1})(d^{\mathtt{T}}\nabla\sigma_{r+1})(v_{r+1}^{\mathtt{T}}d)+\sum_{k=1}^{r}\sigma_{k}(\bar{\mu}^{\mathtt{T}}u_{k})d^{\mathtt{T}}J_{v_{k}}d$. We see from (3.5), that $\theta(t_{r+1},d)$ coincides with $d^{\mathtt{T}}\nabla^{2}_{xx}L(x^{*},\mu)d$ whenever $\mu=\bar{\mu}+\sum_{j=r+1}^{p}t_{j}u_{j}$.

Now, consider the optimization problem:

[TABLE]

From (3.6), we get $\max\{\theta(t_{r+1},d):t_{r+1}\in[a_{*},b_{*}]\}\geq 0$ for all $d\in S(x^{*})$. Observe that $\theta(t,d)$ is linear in $t$ and for each $t$ fixed, it defines a quadratic form as function of $d$. Then, since $\theta(t,d)$ is linear in $t$, the maximum of (3.10) is attained either at $t_{r+1}=a_{*}$ or $t_{r+1}=b_{*}$. For simplicity, let us call the quadratic forms $\theta(a_{*},d)$ and $\theta(b_{*},d)$ by $d^{\mathtt{T}}Pd$ and $d^{\mathtt{T}}Qd$, respectively. Thus, we arrive at

[TABLE]

Applying Yuan’s Lemma (Lemma 2.4), we get the existence of $\alpha\geq 0$ and $\beta\geq 0$ with $\alpha+\beta=1$ such that

[TABLE]

Additionally, due to the linearity of $\theta(t,d)$, we see that $\eta:=\alpha a_{*}+\beta b_{*}$ satisfies $\theta(\eta,d)=d^{\mathtt{T}}(\alpha P+\beta Q)d$. Denote $\pi_{r+1}(t_{r+1},\dots,t_{p}):=t_{r+1}$ the projection onto the first coordinate. From the continuity of $\pi_{r+1}$ and the compactness and convexity of $M$, we get $[a_{*},b_{*}]=\pi_{r+1}(M)$. Since $\eta\in[a_{*},b_{*}]$, we conclude that there are some scalars $\hat{t}_{r+1},\dots,\hat{t}_{p}$ with $\hat{t}_{r+1}=\eta$ and $\hat{\mu}:=\bar{\mu}+\sum_{j=r+1}^{p}\hat{t}_{j}u_{j}\in\Lambda(x^{*})$ such that $\theta(\eta,d)=d^{\mathtt{T}}(\alpha P+\beta Q)d=d^{\mathtt{T}}\nabla^{2}_{xx}L(x^{*},\hat{\mu})d$. From (3.12), we get that $\hat{\mu}$ is a Lagrange multiplier for which WSOC holds at $x^{*}$. ∎

We refer the reader to the companion paper [40] for the proof that our Theorem 3.2 is a generalization of Theorems 2.5 (originally from Baccari and Trad [14]) and 3.1, where the rank deficiency is at most one. It is also a generalization of Theorem 2.7 (originally from Andreani et al. [7]), where the rank is constant, as long as all non-zero singular values are distinct.

In order to see that Theorem 3.2 provides new examples where WSOC holds, let us consider the following:

Example 3.2**.**

[TABLE]

at a local minimizer $x^{*}=(0,0,0)$. The Jacobian matrix for $x$ around $x^{*}$ is given by

[TABLE]

that admits the following smooth SVD decomposition:

[TABLE]

Clearly, MFCQ holds and the rank is $1$ at $x^{*}$ and increases at most to $2$ around $x^{*}$. The set $\Lambda(x^{*})$ of Lagrange multipliers is defined by the relations $\mu_{1}+\mu_{2}+2\mu_{3}=1$ with $\mu_{1},\mu_{2},\mu_{3}\geq 0$. Theorem 3.2 guarantees the existence of a Lagrange multiplier that fulfill WSOC. In fact, we can see that WSOC holds at $x^{*}$ whenever $\mu_{2}\geq\mu_{1}$.

The next example shows, however, that our Theorem 3.2 in the way presented does not prove the complete conjecture, since Assumption 3.1 may fail.

Example 3.3**.**

[TABLE]

at a local minimizer $x^{*}=(0,0,0)$.

The jacobian matrix at $x$ near $x^{*}$ is given by J(x)=\left(\begin{array}[]{ccc}x_{2}&x_{1}&-1\\ -x_{2}&-x_{1}&-1\\ 0&0&-1\end{array}\right). Clearly, MFCQ holds and the rank of $J(x^{*})$ is $1$ and increases at most to $2$ in a neighborhood. Also, WSOC holds. Let us prove that Assumption 3.1 does not hold. Assume that there are differentiable functions $U(x),\Sigma(x),V(x)$ such that $J(x)=U(x)\Sigma(x)V(x)^{\mathtt{T}}$ for all $x$ in a neighborhood of $x^{*}$ as in Assumption 3.1. Let $U(x)=[u_{1},u_{2},u_{3}]$ and $V(x)=[v_{1},v_{2},v_{3}]$ be defined columnwise, where the dependency on $x$ was omitted. Also, let $\sigma_{1},\sigma_{2},\sigma_{3}$ be the diagonal elements of $\Sigma(x)$. Since the rank of $J(x)$ is at most $2$, $\sigma_{3}\equiv 0$. At $x=x^{*}$, since the rank is $1$, we have $\sigma_{1}(x^{*})\neq 0$ and $\sigma_{2}(x^{*})=0$. Then, $u_{1}(x^{*})=\frac{\alpha}{\sigma_{1}(x^{*})}(-1,-1,-1),v_{1}(x^{*})=\frac{1}{\alpha}(0,0,1)$ for some $\alpha\neq 0$ and $u_{2}(x^{*})\perp u_{1}(x^{*})$, $v_{2}(x^{*})\perp v_{1}(x^{*})$. Now, denoting by $[w]_{i},i=1,2,3$ the components of the vector $w\in{\mathbb{R}}^{3}$, the first two columns of the identity $U(x)\Sigma(x)V(x)^{\mathtt{T}}=J(x)$ for all $x$ near $x^{*}$ gives:

[TABLE]

Computing derivatives with respect to $x_{1}$ and $x_{2}$ of every entry, gives, for $i=1,2,3,j=1,2$ and $k=1,2$:

[TABLE]

At $x=x^{*}$, since $\sigma_{2}(x^{*})=0$ and $[v_{1}(x^{*})]_{j}=0,j=1,2$ we have

[TABLE]

For fixed $j=1,2$ and $k=1,2$, we can multiply equation (3.15) by $[u_{1}(x^{*})]_{i}$ and add for $i=1,2,3$ to get:

[TABLE]

Since $\sigma_{1}(x^{*})\neq 0$, computing derivatives, using the definition of $u_{1}(x^{*})$ and the fact that $u_{1}(x^{*})\perp u_{2}(x^{*})$, we conclude that

[TABLE]

Substituting back in (3.15) we have, for all $i=1,2,3,j=1,2,k=1,2$:

[TABLE]

At indices $(i,j,k)\in\{(1,1,2),(1,2,1)\}$, where the right-hand side is non-zero, we have $\frac{\partial\sigma_{2}(x^{*})}{\partial x_{1}}\neq 0,\frac{\partial\sigma_{2}(x^{*})}{\partial x_{2}}\neq 0,[v_{2}(x^{*})]_{1}\neq 0$ and $[v_{2}(x^{*})]_{2}\neq 0$. At indices $(i,j,k)\in\{(1,1,1),(2,1,1),(3,1,1)\}$, where the right-hand side is zero, we get $u_{2}(x^{*})=0$, which is a contradiction.

To conclude this section we note that our proof suggests that when the rank is constant, the Hessian of the Lagrangian does not depend on the Lagrange multiplier. In fact, we can prove this without additional assumptions. This explains why results under constant rank conditions hold for all Lagrange multipliers.

Theorem 3.3**.**

Suppose that $\Lambda(x^{*})\neq\emptyset$. If the rank of $J(x)$ is constant around a point $x^{*}$, then the quadratic form $d^{\mathtt{T}}\nabla^{2}_{xx}L(x^{*},\lambda,\mu)d$ for $d\in S(x^{*})$ does not depend on $(\lambda,\mu)\in\Lambda(x^{*})$.

Proof.

By simplicity, assume $m=0$ and $A(x^{*})=\{1,\dots,p\}$. By Theorem 2.6, for each $d\in S(x^{*})$, there exists a smooth curve $\xi(t)$, $t\in(-\varepsilon,\varepsilon)$ with $g(\xi(t))=0$ for all $t$, with $\xi(0)=x^{*}$ and $\xi^{\prime}(0)=d$. Take $\tilde{\mu}\in\text{Ker}(J(x^{*})^{\mathtt{T}})$ and let us define the function $R(t):=\sum_{i=1}^{p}\tilde{\mu}_{i}g_{i}(\xi(t))$, which is constantly zero for small $t$. Straightforward calculations show that $R^{\prime\prime}(0)=d^{\mathtt{T}}\sum_{i=1}^{p}\tilde{\mu}_{i}\nabla^{2}g_{i}(x^{*})d+\xi^{\prime\prime}(0)^{\mathtt{T}}J(x^{*})^{\mathtt{T}}\tilde{\mu}=0$. Hence, $d^{\mathtt{T}}\sum_{i=1}^{p}\tilde{\mu}_{i}\nabla^{2}g_{i}(x^{*})d=0$.

But $\Lambda(x^{*})=(\bar{\mu}+\text{Ker}(J(x^{*})^{\mathtt{T}}))\cap{\mathbb{R}}^{p}_{+}$ for a fixed Lagrange multiplier $\bar{\mu}\in\Lambda(x^{*})$. Hence $\mu\in\Lambda(x^{*})$ if, and only if, $\mu=\bar{\mu}+\tilde{\mu}$, for some $\tilde{\mu}\in\text{Ker}(J(x^{*})^{\mathtt{T}})$, with $\bar{\mu}+\tilde{\mu}\geq 0$. It follows that $d^{\mathtt{T}}\nabla^{2}_{xx}L(x^{*},\mu)d=d^{\mathtt{T}}\nabla^{2}_{xx}L(x^{*},\bar{\mu})d$, as we wanted to show. Observe that $x^{*}$ is not necessarily a local minimizer, we only require $\Lambda(x^{*})\neq\emptyset$. ∎

Despite our focus on conditions implying WSOC, the above analysis allows us to obtain conclusions about SSOC, related with [14, Theorem 5.1]. Recall that the generalized strict complementary slackness (GSCS) condition holds at the feasible point $x^{*}$ if there exists, at most, one index $i_{*}\in A(x^{*})$ such that $\mu_{i_{*}}=0$ whenever $(\lambda,\mu)\in\Lambda(x^{*})$.

Theorem 3.4**.**

Assume that $x^{*}$ is a local minimizer of (1.1) that fulfills MFCQ. Let $r$ be the rank of $J(x^{*})$, and assume that for every $x$ in some neighborhood of $x^{*}$ the rank of $J(x)$ is at most $r+1$. Suppose also that Assumption 3.1 and GSCS hold at $x^{*}$. Then, there is a Lagrange multiplier ($\lambda,\mu)$ such that SSOC holds.

Proof.

From [14, Theorem 5.1] or [1, Definition 3.3 and Lemma 3.3], it follows that GSCS implies that $C(x^{*})$ is a first-order cone, hence, we can still apply Yuan’s Lemma and prove the result in the same lines of Theorem 3.2. ∎

4 Final remarks

In order to analyse limit points of a sequence generated by a second-order algorithm, one usually relies on WSOC, the stationarity concept based on the critical subspace, the lineality space of the cone of critical directions. Most conditions guaranteeing WSOC at local minimizers are based on a constant rank assumption on the Jacobian matrix. In this paper we developed new tools to deal with the non-constant rank case, by partially solving a conjecture formulated in [7]. Possible future lines of research includes investigating the full conjecture using generalized notions of derivative. We believe this can be done since under the rank assumption, the so-called “crossing” of singular values is controlled, at least when the non-zero ones are simple, which is the main source of non-continuity of singular vectors. Our approach also opens the path to obtaining new second-order results without assuming MFCQ and/or to developing conditions that ensure SSOC, the second-order stationarity concept based on the true critical cone.

Acknowledgements

We thank L. Minchenko for a discussion around Theorem 2.6 after a first draft of this paper was released. This research was funded by FAPESP grants 2013/05475-7 and 2016/02092-8, by CNPq grants 454798/2015-6, 303264/2015-2 and 481992/2013-8, and CAPES.