Necessary Optimality Conditions and Exact Penalization for Non-Lipschitz Nonlinear Programs

TL;DR

This paper extends optimality conditions and exact penalization results to non-Lipschitz nonlinear programs, broadening the theoretical understanding and applicability of KKT conditions and penalty methods in such challenging settings.

Contribution

It introduces generalized constraint qualifications and establishes exact penalization results for non-Lipschitz objectives, expanding the scope of optimality theory.

Findings

Extended quasi-normality and RCPLD conditions ensure KKT necessity for non-Lipschitz problems.

Derived exact penalization results for problems with composite and indicator functions.

Provided conditions under which local minimizers coincide with penalized problem solutions.

Abstract

When the objective function is not locally Lipschitz, constraint qualifications are no longer sufficient for Karush-Kuhn-Tucker (KKT) conditions to hold at a local minimizer, let alone ensuring an exact penalization. In this paper, we extend quasi-normality and relaxed constant positive linear dependence (RCPLD) condition to allow the non-Lipschitzness of the objective function and show that they are sufficient for KKT conditions to be necessary for optimality. Moreover, we derive exact penalization results for the following two special cases. When the non-Lipschitz term in the objective function is the sum of a composite function of a separable lower semi-continuous function with a continuous function and an indicator function of a closed subset, we show that a local minimizer of our problem is also a local minimizer of an exact penalization problem under a local error bound condition for a restricted constraint region and a suitable assumption on the outer separable function. When the non-Lipschitz term is the sum of a continuous function and an indicator function of a closed subset, we also show that our problem admits an exact penalization under an extended quasi-normality involving the coderivative of the continuous function.

Equations

$$\min$$

$$s.t.$$

$$\displaystyle\left\{\begin{array}[]{ll}0\in\partial^{\infty}\Phi(x^{*})+\sum\limits_{i=1}^{n}\lambda_{i}\partial g_{i}(x^{*})+\sum\limits_{j=1}^{m}\partial(\mu_{j}h_{j})(x^{*})\\[2.0pt] \lambda_{i}\geq 0,\ \lambda_{i}g_{i}(x^{*})=0\ i=1,\ldots,n\end{array}\right.\Longrightarrow(\lambda,\mu)=0,$$

$$\hat{\partial }\phi (x^{\ast }):=\{v:\phi (x)\ge \phi (x^{\ast })+v^T(x-x^{\ast })+o(\Vert x-x^{\ast }\Vert )\forall x\},$$

$$\partial \phi (x^{\ast }):=\{v:\exists x^k{\to }_{\phi }{x}^{\ast },v^k\in \hat{\partial }\phi (x^k)s.t.v^k\to v\},$$

$${\partial }^{\infty }\phi (x^{\ast }):=\{v:\exists x^k{\to }_{\phi }{x}^{\ast },v^k\in \hat{\partial }\phi (x^k)\mathrm{and}{t}_k\to 0\mathrm{with}{t}_k\ge 0s.t.t_k{v}^k\to v\},$$

$$D^{\ast }\mathcal{S}(\bar{x}|\bar{u})(y):=\{v:(v,-y)\in {\mathcal{N}}_{\mathrm{gph}\mathcal{S}}(\bar{x},\bar{u})\},$$

$$D^{\ast }\mathcal{S}(\bar{x})(y)=\partial ⟨y,\mathcal{S}⟩(\bar{x})\forall y\in {\Re }^m.$$

$$\Vert v\Vert \le M\forall v\in \mathcal{S}(x),\forall x\in {\mathcal{B}}_{\delta }(\bar{x}).$$

$$\{\bar{v}:\exists x^k\to \bar{x},v^k\in \mathcal{S}(x^k)s.t.v^k\to \bar{v}\}\subseteq \mathcal{S}(\bar{x}).$$

$$\partial (\alpha f+\beta g)(x^{\ast })\subseteq \alpha \partial f(x^{\ast })+\beta \partial g(x^{\ast }).$$

$$\partial (f\circ g)(x^{\ast })\subseteq \bigcup _{\xi \in \partial f(g(x^{\ast }))}\partial ⟨\xi ,g⟩(x^{\ast }).$$

$$\partial (f\circ g)(x^{\ast })\subseteq \bigcup _{\xi \in \partial f(g(x^{\ast }))}{D}^{\ast }g(x^{\ast })(\xi ),{\partial }^{\infty }(f\circ g)(x^{\ast })\subseteq D^{\ast }g(x^{\ast })(0).$$

$$\partial f(x^{\ast })\subseteq \bigcup \left\{\sum _{i\in {\mathcal{I}}^{\ast }}{\lambda }_i\diamond \partial f_i(x^{\ast }):({\lambda }_1,\dots ,{\lambda }_s)\in {\Lambda }^{\ast }\right\},$$

$${\Lambda }^{\ast }:=\left\{({\lambda }_1,\dots ,{\lambda }_s):{\lambda }_i\ge 0i\in {\mathcal{I}}^{\ast },{\lambda }_i=0i\notin {\mathcal{I}}^{\ast },\sum _{i\in {\mathcal{I}}^{\ast }}{\lambda }_i=1\right\}.$$

$$0\in {\partial }^{\infty }\Phi (x)+\sum _{i=1}^n{\lambda }_i\partial g_i(x)+\sum _{j=1}^m\partial ({\mu }_j{h}_j)(x),$$

$${\lambda }_i\ge 0,{\lambda }_i{g}_i(x)=0i=1,\dots ,n,$$

$$g_i(x^k)>0i\in I,{\mu }_j{h}_j(x^k)>0j\in J,\forall k\in \mathbb{N},$$

$$0\in {\partial }^{\infty }\Phi (x)+\sum _{i\in \mathcal{I}}{\lambda }_i\nabla g_i(x)+\sum _{j\in \mathcal{J}}{\mu }_j\nabla h_j(x),$$

$$0\in \sum _{i\in \mathcal{I}}{\lambda }_i\nabla g_i(x)+\sum _{j\in \mathcal{J}}{\mu }_j\nabla h_j(x)+{\mathcal{N}}_{\Omega }(x).$$

$$0\in D^{\ast }\Psi (x)(0)+\sum _{i=1}^n{\lambda }_i\partial g_i(x)+\sum _{j=1}^m\partial ({\mu }_j{h}_j)(x)+{\mathcal{N}}_{\Omega }(x).$$

$$\displaystyle\begin{array}[]{rl}\min&f(x):=\sqrt{|x_{1}|}+\sqrt{|x_{2}|}+\sqrt{|x_{3}|}+\sqrt{|x_{4}|}\\[3.0pt] {\rm s.t.}&g_{1}(x):=x_{1}+x_{2}+x_{3}+x_{4}-2\leq 0,\\[2.0pt] &h_{1}(x):=x_{1}+x_{2}-1=0,\\[2.0pt] &h_{2}(x):=x_{3}+x_{4}-1=0\end{array}$$

$$0\in {\partial }^{\infty }f(x^{\ast })+{\lambda }_1\nabla g_1(x^{\ast })+{\mu }_1\nabla h_1(x^{\ast })+{\mu }_2\nabla h_2(x^{\ast }),{\lambda }_1\ge 0,$$

$$g_1(x^k)>0\mathrm{if}{\lambda }_1>0,{\mu }_1{h}_1(x^k)>0\mathrm{if}{\mu }_1\ne 0,{\mu }_2{h}_2(x^k)>0\mathrm{if}{\mu }_2\ne 0.$$

$$2<x_1^k+x_2^k+x_3^k+x_4^k<2.$$

$$\displaystyle\begin{array}[]{rl}\min&f(x):=\sqrt{|x_{1}|}+\sqrt{|x_{2}|}+\sqrt{|x_{3}|}\\[3.0pt] {\rm s.t.}&g(x):=x_{1}+x_{2}-x_{3}^{2}-1\leq 0,\\[2.0pt] &h(x):=x_{1}+x_{2}-1=0\end{array}$$

$$0\in {\partial }^{\infty }f(x^{\ast })+\lambda \nabla g(x^{\ast })+\mu \nabla h(x^{\ast }),\lambda \ge 0$$

$$0\in {\partial }^{\infty }f(x^{\ast })+\lambda \nabla g(x^{\ast })+\mu \nabla h(x^{\ast }),\lambda \ge 0,$$

$$g(x^k)>0\mathrm{if}\lambda >0,\mu h(x^k)>0\mathrm{if}\mu \ne 0.$$

$$\min f(x)s.t.x\in \mathcal{F}$$

$$\min f(x)+\sqrt{|z|}s.t.x\in \mathcal{F},z\in \Re ,$$

$$-{\partial }^{\infty }\Phi (x^{\ast })\cap {\mathcal{N}}_{\mathcal{F}}(x^{\ast })=\{0\}.$$

Full text

††journal: Mathematical Programming

1 Introduction

The purpose of this paper is to study necessary optimality conditions and exact penalization for the following non-Lipschitz nonlinear program:

[TABLE]

where $f:\Re^{d}\to\Re,g:\Re^{d}\to\Re^{n},h:\Re^{d}\to\Re^{m}$ are Lipschitz around the point of interest, and $\Phi:\Re^{d}\to(-\infty,\infty]$ is an extended-valued lower semi-continuous function.

Including a non-Lipschitz term in the objective function has significantly enlarged the applicability of standard nonlinear programs. For example, it has recently been discovered that when the term $\Phi$ belongs to a certain class of non-Lipschitz functions, local minimizers of problem (1) are often sparse. This property makes problem (1) useful for seeking a sparse solution in many fields such as image restoration, signal processing, wireless communication, and portfolio selection in financial applications; see, e.g., int:Donoho ; Chartrand07 ; BianChen15 ; Chen2013 ; Liu2015 .

It is well-known that a constraint qualification is a condition imposed on constraint functions so that Karush-Kuhn-Tucker (KKT) conditions hold at a local minimizer. There exist very weak constraint qualifications such as Guignard’s and Abadie’s constraint qualifications (Guignard ; Abadie ) but they are not easy to verify since it involves computing the tangent or normal cone of the constraint region. The challenge is to find verifiable constraint qualifications that are applicable to as many situations as possible. For nonlinear programs where the objective function is locally Lipschitz, the verifiable classical constraint qualifications in the literature include linear independence constraint qualification, Slater’s condition, and Mangasarian-Fromovitz constraint qualification (MFCQ). Moreover, it is well-known that when all constraint functions are linear, no constraint qualification is required for KKT conditions to hold at a local minimizer. In recent years, quite a few new and weaker verifiable constraint qualifications have been introduced; see, e.g., BertsekasJOTA2002 ; cpld-quasi ; gfrerer ; Minchenko-stakhovski ; RCPLD ; abdeabi–twoCQ ; ccp-2016 ; SCQ . In particular, quasi-normality is a weak constraint qualification that was first introduced in BertsekasJOTA2002 and extended to locally Lipschitz programs in ye-zhang . The recently introduced relaxed constant positive linear dependence (RCPLD) condition in RCPLD is also a weak constraint qualification. Both quasi-normality and RCPLD are weaker than MFCQ and hold automatically when all constraint functions are linear (see (BertsekasJOTA2002, , Proposition 3.1)). Moreover, they can admit a local error bound for the constraint region (see RCPLD ; guoyezhang-infinite ) and thus by Clarke’s exact penalization principle (c, , Proposition 2.4.3), they are sufficient to ensure an exact penalization when the objective function is locally Lipschitz.

Little research has been done in KKT necessary optimality conditions for non-Lipschitz nonlinear programs in the literature, let alone exact penalization. The Fritz John type necessary optimality conditions for non-Lipschitz programs were first given by Kruger and Mordukhovich in Kruger-Mord79 and reproved by Mordukhovich in (Mor80, , Theorem 1(b)) and Borwein et al. in (int:Borwein, , Corollary 2.6). For our problem, since all functions are locally Lipschitz except the objective function, (A1) in (int:Borwein, , Corollary 2.6) never holds and consequently (A2) in (int:Borwein, , Corollary 2.6) holds. Hence, the Fritz John condition (int:Borwein, , Corollary 2.6) for our problem states that at a local minimizer $x^{*}$, there exist $\lambda\in\Re^{|{\cal I}^{*}|}_{+}$ and $\mu\in\Re^{m}$ not all zero such that at least one of the following cases holds:

• (i)

$0\in\partial^{\infty}\Phi(x^{*})+\sum\limits_{i\in{\cal I}^{*}}\lambda_{i}\partial g_{i}(x^{*})+\sum\limits_{j=1}^{m}\partial(\mu_{j}h_{j})(x^{*})$,

• (ii)

$0\in\partial(f+\Phi)(x^{*})+\sum\limits_{i\in{\cal I}^{*}}\lambda_{i}\partial g_{i}(x^{*})+\sum\limits_{j=1}^{m}\partial(\mu_{j}h_{j})(x^{*})$,

where ${\cal I}^{*}:=\{i:g_{i}(x^{*})=0\}$, and $\partial,\partial^{\infty}$ denote the limiting subdifferential and the horizon subdifferential respectively (see the definitions in Section 2). Consequently, we can derive the KKT necessary optimality condition from the above Fritz John condition immediately as follows. Suppose that there are no nonzero abnormal multipliers, i.e., the following implication holds:

[TABLE]

then the above condition (ii) holds, which means that $x^{*}$ is a KKT point. We call the implication (4) $\partial^{\infty}$-no nonzero abnormal multiplier constraint qualification ($\partial^{\infty}$-NNAMCQ) at $x^{*}$. Note that when $\Phi$ is Lipschitz around $x^{*}$, we have $\partial^{\infty}\Phi(x^{*})=\{0\}$ and hence $\partial^{\infty}$-NNAMCQ reduces to the standard NNAMCQ for nonlinear programs with equality and inequality constraints. When $\Phi$ is an indicator function of a closed subset, $\partial^{\infty}$-NNAMCQ reduces to the standard NNAMCQ for nonlinear programs with equality, inequality, and abstract set constraints. When $\Phi$ is neither Lipschitz around $x^{*}$ nor equal to an indicator function, the implication (4) involves the horizon subdifferential $\partial^{\infty}\Phi(x^{*})$ of the non-Lipschitz term. Thus, $\partial^{\infty}$-NNAMCQ is no longer a constraint qualification since it is related to the objective function. However, since it is a condition under which a local minimizer is a KKT point, we call such a condition a qualification condition. Very recently, Chen et al. chenguoluye gave some necessary optimality conditions for problem (1) where the non-Lipschitz term $\Phi$ is continuous and all the other functions are continuously differentiable under RCPLD, and the so-called basic qualification (BQ for short) (see the definition in (16)), and proposed an augmented Lagrangian method for solving this kind of problems. It should be noted that BQ is very difficult to verify as discussed in the paragraph after Corollary 1.

In this paper, we extend the standard quasi-normality and the standard RCPLD to problem (1). Similar to $\partial^{\infty}$-NNAMCQ, our new qualification conditions also involve $\partial^{\infty}\Phi(x^{*})$ and we call them $\partial^{\infty}$-quasi-normality and $\partial^{\infty}$-RCPLD respectively. Moreover, we derive two exact penalization results for two special cases of problem (1) under some suitable conditions. We summarize our main contributions as follows:

• •

We introduce two new verifiable qualification conditions called $\partial^{\infty}$-quasi-normality and $\partial^{\infty}$-RCPLD respectively and show that they are sufficient for KKT conditions to be necessary for optimality. These two qualification conditions are both weaker than $\partial^{\infty}$-NNAMCQ and hold automatically when $\Phi$ is Lipschitz around the point of interest and $g,h$ are linear. As a by-product, we extend the standard RCPLD on smooth constraint functions to the case where there is an extra abstract set constraint and show that KKT conditions are necessary for optimality.

• •

Exact penalization for two special cases of problem (1) are derived. Case i): $\Phi$ is the sum of a composite function of a separable lower semi-continuous function with a continuous function and an indictor function of a closed subset. In this case, we show that a local minimizer of problem (1) is also a local minimizer of an exact penalization problem under a local error bound condition for a restricted constraint region and a suitable assumption on the outer separable function. Case ii): $\Phi$ is the sum of a continuous function and an indicator function of a closed subset. In this case, we introduce $D^{*}$-quasi-normality that is an extended quasi-normality involving the coderivative of the continuous function, and show that $D^{*}$-quasi-normality is sufficient to ensure an exact penalization. Note that $D^{*}$-quasi-normality reduces to the standard quasi-normality for nonlinear programs with equality, inequality, and abstract set constraints when the continuous function is Lipschitz around the point of interest.

The rest of this paper is organized as follows. In Section 2 we give some background materials. In Section 3 we propose some qualification conditions for problem (1). In Section 4 we derive necessary optimality conditions for problem (1) under these qualification conditions. We investigate some sufficient conditions ensuring an exact penalization for problem (1) in Section 5.

2 Preliminaries

The notations used in this paper are standard in the literature. The symbol ${\mathbb{N}}$ (resp., $\Re,\Re_{+},\Re_{-}$) denotes the set of nonnegative integers (resp., real numbers, nonnegative real numbers, nonpositive real numbers). For a finite set $T$, $|T|$ denotes its cardinality. For any $x\in\Re^{d}$, we denote by $x_{+}:=\max\{x,0\}$ the non-negative part of $x$, $\|x\|_{p}:={\large(\sum\limits_{i=1}^{d}|x_{i}|^{p}\large)^{1/p}}$ for any $p>0$, and $\|x\|$ any norm in $\Re^{d}$. Let ${\cal B}_{\delta}(x)$ denote a closed ball centered at $x$ with positive radius $\delta$. The indicator function of a subset ${\cal D}\subseteq\Re^{d}$ is denoted by $\delta_{\cal D}$ and ${\mathop{\rm dist}}_{\cal D}(x)$ denotes the Euclidean distance from $x$ to ${\cal D}$. Let ${\cal F}$ denote the constraint region for problem (1) and for any $x\in{\cal F}$, denote by ${\cal I}_{g}(x):=\{i:g_{i}(x)=0\}$ the index set of active inequality constraints.

We say that ${\cal F}$ admits a local error bound at ${\bar{x}}\in{\cal F}$ if there exist $\delta>0$ and $\kappa>0$ such that ${\mathop{\rm dist}}_{\cal F}(x)\leq\kappa(\|h(x)\|+\|g(x)_{+}\|)$ for all $x\in{\cal B}_{\delta}({\bar{x}})$.

We next give some background materials on variational analysis; see, e.g., c ; clsw ; Rock98 ; mor1 for more details. For a function $\varphi:\Re^{d}\to[-\infty,\infty]$ and a point $x^{*}\in\Re^{d}$ where $\varphi(x^{*})$ is finite, the regular (or Fréchet) subdifferential of $\varphi$ at $x^{*}$ is defined as

[TABLE]

the limiting (or Mordukhovich) subdifferential of $\varphi$ at $x^{*}$ is defined as

[TABLE]

and the horizon (or singular Mordukhovich) subdifferential of $\varphi$ at $x^{*}$ is defined as

[TABLE]

where $o(\cdot)$ means $o(\alpha)/\alpha\to 0$ as $\alpha\to 0$, and $x^{k}\to_{\varphi}x^{*}$ means $x^{k}\to x^{*}$ and $\varphi(x^{k})\to\varphi(x^{*})$ as $k\to\infty$. It is well-known that $\varphi$ is Lipschitz around $x^{*}$ if and only if $\partial^{\infty}\varphi(x^{*})=\{0\}$ by (Rock98, , Theorem 9.13).

The regular (or Fréchet) normal cone of ${\cal D}$ at $x^{*}\in{\cal D}$ is a closed convex cone defined as $\widehat{{\cal N}}_{\cal D}(x^{*}):=\hat{\partial}\delta_{\cal D}(x^{*})$ and the limiting (or Mordukhovich) normal cone of ${\cal D}$ at $x^{*}$ is a closed cone defined as ${\cal N}_{\cal D}(x^{*}):=\partial\delta_{\cal D}(x^{*})$. We say that ${\cal D}$ is regular at $x^{*}$ if ${\cal D}$ is locally closed at $x^{*}$ and ${\cal N}_{\cal D}(x^{*})=\widehat{{\cal N}}_{\cal D}(x^{*})$.

Given a set-valued mapping ${\cal S}:\Re^{d}\rightrightarrows\Re^{m}$ and a point $\bar{x}$ with ${\cal S}(\bar{x})\neq\emptyset$, the coderivative of ${\cal S}$ at $\bar{x}$ for any $\bar{u}\in{\cal S}(\bar{x})$ is the mapping $D^{*}{\cal S}(\bar{x}|\bar{u}):\Re^{m}\rightrightarrows\Re^{d}$ defined by

[TABLE]

where ${\rm gph\,{\cal S}}:=\{(x,y):y\in{\cal S}(x)\}$. When ${\cal S}$ is single-valued at $\bar{x}$ with ${\cal S}(\bar{x})=\bar{u}$, the notation $D^{*}{\cal S}(\bar{x}|\bar{u})$ is simplified to $D^{*}{\cal S}(\bar{x})$. In the case where ${\cal S}$ is not only single-valued but also Lipschitz around $\bar{x}$, the coderivative is related to the limiting subdifferential by the scalarization formula:

[TABLE]

We say that ${\cal S}$ is locally bounded at $\bar{x}\in\Re^{d}$ if there exist $M>0$ and $\delta>0$ such that

[TABLE]

Recall from (Rock98, , Definition 5.4) that ${\cal S}$ is said to be outer semi-continuous at $\bar{x}$ if

[TABLE]

It is well-known that the limiting normal cone mapping, the limiting subdifferential mapping, and the horizon subdifferential mapping are all outer semi-continuous everywhere; see, e.g., (Rock98, , Propositions 6.6 and 8.7).

By using the outer semi-continuity of the limiting normal cone mapping and the definition of the coderivative, it is easy to give the following proposition that will be useful in deriving exact penalization results in Section 5.

Proposition 1

The coderivative $D^{*}S(x|u):\Re^{d}\rightrightarrows\Re^{m}$ is outer semi-continuous in the sense that if there exists $v^{k}\in D^{*}S(x^{k}|u^{k})(y^{k})$ where $x^{k}\to x^{*}$, $y^{k}\to y^{*}$, and $u^{k}\to u^{*}$ with $u^{k}\in S(x^{k})$ such that $v^{k}\to v^{*}$, then $v^{*}\in D^{*}S(x^{*}|u^{*})(y^{*})$.

The following proposition collects some useful properties and calculus rules of the limiting subdifferential.

Proposition 2

• (i)

(Rock98, , Exercise 10.10)* Let $f,g:\Re^{d}\to[-\infty,\infty]$ be proper lower semi-continuous around $x^{*}\in\Re^{d}$ and finite at $x^{*}$, and let $\alpha,\beta$ be nonnegative scalars. Assume that at least one of them is Lipschitz around $x^{*}$. Then*

[TABLE]

Here we let $0\cdot\emptyset=\{0\}$ by convention.

• (ii)

(jourani1993, , Theorem 2.5 and Remark (2))* Let $g:\Re^{n}\to\Re^{m}$ be Lipschitz around $x^{*}$ and $f:\Re^{m}\to\Re$ be Lipschitz around $g(x^{*})$. Then the composite function $f\circ g$ is Lipschitz around $x^{*}$ and*

[TABLE]

• (iii)

(mor1, , Theorem 3.38)* Let $g:\Re^{d}\to\Re^{m}$ be continuous at $x^{*}$ and $f:\Re^{m}\to\Re$ be Lipschitz around $g(x^{*})$. Then*

[TABLE]

• (iv)

(Mor-nonsmooth, , Theorem 7.5)* Let $f({x}):=\max\{f_{i}({x}):i=1,\ldots,s\}$ where $f_{i}:\Re^{d}\to\Re$ is continuous at $x^{*}$ for all $i=1,\dots,s$. If all but at most one of the functions $\{f_{i}:i=1,\ldots,s\}$ are Lipschitz around $x^{*}$, then*

[TABLE]

where ${\cal I}^{*}:=\{i:f_{i}(x^{*})=f(x^{*})\}$ is the index set of active indices and

[TABLE]

Here we define $\alpha\diamond\partial g$ by $\alpha\partial g$ if $\alpha>0$ and by $\partial^{\infty}g$ if $\alpha=0$.

3 Qualification conditions

Since the objective function of problem (1) includes a non-Lipschitz term, KKT conditions are no longer necessary for optimality only under constraint qualifications such as the standard quasi-normality (ye-zhang, , Definition 5) and the standard RCPLD (RCPLD, , Definition 4). In this section we extend the standard quasi-normality and the standard RCPLD to problem (1) as follows so that KKT conditions can be necessary for optimality under the extended quasi-normality and the extended RCPLD respectively.

Definition 1

Let $x\in{\cal F}$. (a) We say that $x$ is $\partial^{\infty}$-quasi-normal if there is no nonzero vector $(\lambda,\mu)\in\Re^{n}\times\Re^{m}$ such that there exists a sequence $\{x^{k}\}$ which converges to $x$ as $k\to\infty$ satisfying

[TABLE]

where $I:=\{i:\lambda_{i}>0\}$, $J:=\{j:\mu_{j}\neq 0\}$, and ${\mathbb{N}}$ is the set of all positive integers.

(b) Assume that $g,h$ are smooth around $x$. Let ${\cal J}\subseteq\{1,\ldots,m\}$ be such that $\{\nabla h_{j}(x):j\in{\cal J}\}$ is a basis for the ${\rm span}\,\{\nabla h_{j}(x):j=1,\ldots,m\}$. We say that $\partial^{\infty}$-RCPLD condition holds at $x$ if there exists $\delta>0$ such that

• (i)

$\{\nabla h_{j}(y):j=1,\ldots,m\}$ has the same rank for each $y\in{\cal B}_{\delta}(x)$;

• (ii)

for each ${\cal I}\subseteq{\cal I}_{g}(x)$, if there exist $\{\lambda_{i}\geq 0:i\in{\cal I}\}$ and $\{\mu_{j}:j\in{\cal J}\}$ not all zero such that

[TABLE]

then $\{\nabla g_{i}(y),\nabla h_{j}(y):i\in{\cal I},j\in{\cal J}\}$ is linearly dependent for each $y\in{\cal B}_{\delta}(x)$.

It is easy to see that both $\partial^{\infty}$-quasi-normality and $\partial^{\infty}$-RCPLD are weaker than $\partial^{\infty}$-NNAMCQ (i.e., implication (4)) but the reverse is not true; see Examples 1–2. Note that if $\Phi$ is Lipschitz around $x$, then $\partial^{\infty}\Phi(x)=\{0\}$, and thus $\partial^{\infty}$-quasi-normality and $\partial^{\infty}$-RCPLD reduce to the standard quasi-normality and the standard RCPLD respectively for nonlinear programs with equality and inequality constraints. If $\Phi$ is an indicator function of a closed subset $\Omega$, i.e., $\Phi(x)=\delta_{\Omega}(x)$, then $\partial^{\infty}\Phi(x)={\cal N}_{\Omega}(x)$ by (Rock98, , Exercise 8.14). Thus $\partial^{\infty}$-quasi-normality reduces to the standard quasi-normality for nonlinear programs with equality, inequality, and abstract set constraints, and $\partial^{\infty}$-RCPLD allows us to extend the original definition of RCPLD (RCPLD, , Definition 4) to the problem where there is an extra abstract set constraint $x\in\Omega$ since inclusion (8) becomes

[TABLE]

In this case we simply say that RCPLD holds.

We next extend the standard quasi-normality to problem (1) for ensuring an exact penalization.

Definition 2

Suppose that $\Phi(x):=\Psi(x)+\delta_{\Omega}(x)$ where $\Psi$ is a continuous function and $\Omega$ is a closed subset in $\Re^{d}$. Let $x\in{{\cal F}}$. We say that $x$ is $D^{*}$-quasi-normal if there is no nonzero vector $(\lambda,\mu)\in\Re^{n}\times\Re^{m}$ such that there exists a sequence $\{x^{k}\}$ which converges to $x$ as $k\to\infty$ satisfying (6)–(7) and

[TABLE]

If $\Psi$ is Lipschitz around $x$, then $D^{*}\Psi(x)(0)=\{0\}$ and hence $D^{*}$-quasi-normality reduces to the standard quasi-normality for nonlinear programs with equality, inequality, and abstract set constraints. Since $\partial^{\infty}\Phi(x)\subseteq D^{*}\Phi(x)(0)$ (see (mor1, , Theorem 1.80)), $D^{*}$-quasi-normality is stronger than $\partial^{\infty}$-quasi-normality when $\Omega=\Re^{d}$.

We call problem (1) an $\ell_{1/2}$ minimization problem if the non-Lipschitz term $\Phi(x)$ is equal to $(\|x\|_{1/2})^{1/2}$. The problem in the following example is an $\ell_{1/2}$ minimization problem with linear constraints. It gives an example for which $\partial^{\infty}$-RCPLD, $\partial^{\infty}$-quasi-normality, and $D^{*}$-quasi-normality are all satisfied but $\partial^{\infty}$-NNAMCQ does not hold.

Example 1

Consider the following problem

[TABLE]

at a minimizer $x^{*}={(1,0,1,0)}$. By direct calculation, we have $\partial^{\infty}f(x^{*})=\{0\}\times\Re\times\{0\}\times\Re$, $\nabla g_{1}(x)={(1,1,1,1)}$, $\nabla h_{1}(x)={(1,1,0,0)}$, and $\nabla h_{2}(x)={(0,0,1,1)}$ for any $x$. Direct verification implies that there exists $(\lambda_{1},\mu_{1},\mu_{2})\neq 0$ such that

[TABLE]

which means that $\partial^{\infty}$-NNAMCQ does not hold at $x^{*}$. But in this case, the family of gradients $\{\nabla g_{1}(x),\nabla h_{1}(x),\nabla h_{2}(x)\}$ is linearly dependent for any $x$. Thus, $\partial^{\infty}$-RCPLD holds at $x^{*}$. To show that $\partial^{\infty}$-quasi-normality also holds at $x^{*}$, we assume that there exist $(\lambda_{1},\mu_{1},\mu_{2})\neq 0$ and a sequence $\{x^{k}\}$ converging to $x^{*}$ such that

[TABLE]

By (10), it follows that $\mu_{1}=\mu_{2}=-\lambda_{1}$. Thus $\lambda_{1}>0$ and $\mu_{1}=\mu_{2}<0$. These together with (11) leads to

[TABLE]

This contradiction shows that there is no nonzero $(\lambda_{1},\mu_{1},\mu_{2})$ satisfying (10)–(11). Thus, $\partial^{\infty}$-quasi-normality holds at $x^{*}$. Since it is easy to verify that $D^{*}f(x^{*})(0)=\partial^{\infty}f(x^{*})$, $D^{*}$-quasi-normality holds at $x^{*}$ as well.   $\square$

The following example of an $\ell_{1/2}$ minimization problem with nonlinear constraints illustrates that it is possible that $\partial^{\infty}$-quasi-normality holds but $\partial^{\infty}$-RCPLD does not hold.

Example 2

Consider the following problem

[TABLE]

at a minimizer $x^{*}={(1,0,0)}$. $\partial^{\infty}$-RCPLD does not hold at $x^{*}$ since there exists $(\lambda,\mu)\neq 0$ such that

[TABLE]

but $\{\nabla g(x),\nabla h(x)\}$ is linearly independent for any $x$ with $x_{3}\not=0$. To show that $\partial^{\infty}$-quasi-normality holds at $x^{*}$, assume that there exist $(\lambda,\mu)\neq 0$ and a sequence $\{x^{k}\}$ converging to $x^{*}$ such that

[TABLE]

By (13), it follows that $\lambda+\mu=0$. Thus $\lambda>0$ and $\mu<0$. But by (14), we have $1+(x_{3}^{k})^{2}<1$. The contradiction shows that $\partial^{\infty}$-quasi-normality holds at $x^{*}$.   $\square$

We know that the standard RCPLD does not imply the standard quasi-normality (see (RCPLD, , Example 2)). It is also easy to see that the standard RCPLD and the standard quasi-normality at a local minimizer $x^{*}$ of the problem

[TABLE]

are equivalent to $\partial^{\infty}$-RCPLD and $\partial^{\infty}$-quasi-normality at a local minimizer $(x^{*},0)$ of the perturbed problem

[TABLE]

respectively. Thus, if problem (15) is such that the standard RCPLD holds but the standard quasi-normality does not hold, then $\partial^{\infty}$-RCPLD holds but $\partial^{\infty}$-quasi-normality does not hold for the above perturbed problem.

We next give some characterizations for $\partial^{\infty}$-quasi-normality, $\partial^{\infty}$-RCPLD, and $D^{*}$-quasi-normality in terms of the standard quasi-normality and the standard RCPLD.

Proposition 3

Let $x^{*}\in{\cal F}$. (i) If $\partial^{\infty}$-quasi-normality holds at $x^{*}$, then both the standard quasi-normality and the following basic qualification (BQ) hold at $x^{*}$:

[TABLE]

If $g,h$ are smooth around $x^{*}$, then $\partial^{\infty}$-quasi-normality at $x^{*}$ is equivalent to the standard quasi-normality plus BQ (16) at $x^{*}$.

(ii)* $\partial^{\infty}$-RCPLD holds at $x^{*}$ if and only if both the standard RCPLD and BQ (16) hold at $x^{*}$.*

(iii)* If $D^{*}$-quasi-normality holds at $x^{*}$, then the standard quasi-normality holds at $x^{*}$ and*

[TABLE]

If $\Omega$ is regular and $g,h$ are smooth around $x^{*}$, then $D^{*}$-quasi-normality holds at $x^{*}$ if and only if both the standard quasi-normality and condition (17) hold at $x^{*}$.

Proof

(i) Suppose that $\partial^{\infty}$-quasi-normality holds at $x^{*}$. Then it is easy to see that the standard quasi-normality also holds at $x^{*}$ since $0\in\partial^{\infty}\Phi(x^{*})$. Thus by (ye-zhang, , Proposition 4), it follows that

[TABLE]

where

[TABLE]

We now show that BQ (16) holds. By contradiction, suppose that $0\neq\zeta\in-\partial^{\infty}\Phi(x^{*})\cap{\cal N}_{\cal F}(x^{*})$. Then by (18), there exist $(\lambda,\mu)\in\Re^{|{\cal I}^{*}|}\times\Re^{m}$ and a sequence $\{x^{k}\}$ converging to $x^{*}$ such that

[TABLE]

Since $0\neq\zeta\in-\partial^{\infty}\Phi(x^{*})$, it then follows that $(\lambda,\mu)\neq 0$ and

[TABLE]

which contradicts $\partial^{\infty}$-quasi-normality. Thus, BQ (16) holds.

We next show the converse part. Assume that $g,h$ are smooth around $x^{*}$, and both the standard quasi-normality and BQ (16) hold at $x^{*}$. By contradiction, assume that $\partial^{\infty}$-quasi-normality does not hold at $x^{*}$. That is, there exist $0\neq(\lambda,\mu)\in\Re^{|{\cal I}^{*}|}\times\Re^{m}$ and a sequence $\{x^{k}\}$ converging to $x^{*}$ such that

[TABLE]

where ${\cal I}^{*},I,J$ are defined as in (19). Moreover, since $g,h$ are smooth around $x^{*}$, it follows from (Rock98, , Theorem 6.14) that

[TABLE]

This and (22) imply that

[TABLE]

which together with BQ (16) means that

[TABLE]

This together with (23) and the relation $(\lambda,\mu)\neq 0$ contradicts the standard quasi-normality. Thus, $\partial^{\infty}$-quasi-normality holds at $x^{*}$.

(ii) Let the standard RCPLD and BQ (16) hold at $x^{*}$. Let ${\cal J}$ be the index set given in the definition of the standard RCPLD such that $\{\nabla h_{j}(x^{*}):j\in{\cal J}\}$ is a basis for the ${\rm span}\,\{\nabla h_{j}(x^{*}):j=1,\ldots,m\}$, and let ${\cal I}^{*}$ be defined as in (19). To show $\partial^{\infty}$-RCPLD, it suffices to show that Definition 1(b)(ii) holds. Assume that there exist nonzero vectors $\{\alpha_{i}\geq 0:i\in{\cal I}\}$ with ${\cal I}\subseteq{\cal I}^{*}$ and $\{\beta_{j}:j\in{\cal J}\}$ such that

[TABLE]

which together with (16) and (24) implies that

[TABLE]

This and the standard RCPLD imply the existence of $\delta>0$ such that

[TABLE]

Thus, Definition 1(b)(ii) holds and then $\partial^{\infty}$-RCPLD holds at $x^{*}$.

To show the converse part, suppose that $\partial^{\infty}$-RCPLD holds at $x^{*}$. It then follows immediately that RCPLD holds at $x^{*}$ since $0\in\partial^{\infty}\Phi(x^{*})$. Then by (Guo14, , Theorem 3.2), it follows that

[TABLE]

We now show that BQ (16) holds. To the contrary, assume that $0\neq\zeta\in-\partial^{\infty}\Phi(x^{*})\cap{\cal N}_{\cal F}(x^{*})$. Let ${\cal J}$ be such that $\{\nabla h_{j}(x^{*}):j\in{\cal J}\}$ is a basis for the ${\rm span}\,\{\nabla h_{j}(x^{*}):j=1,\ldots,m\}$. Then by (25), there exist $\{\lambda_{i}>0:i\in{\cal I}\}$ with ${\cal I}\subseteq{\cal I}^{*}$ and $\{\mu_{j}:j\in{\cal J}\}$ not all zero such that

[TABLE]

which together with the relation $\zeta\in-\partial^{\infty}\Phi(x^{*})$ implies that

[TABLE]

Then by Definition 1(b)(ii), there exist $\{\alpha_{i}:i\in{\cal I}\}$ and $\{\beta_{j}:j\in{\cal J}\}$ not all zero such that

[TABLE]

which together with (26) implies that for any $\gamma\in\Re$,

[TABLE]

Choosing $\gamma\neq 0$ as the smallest number such that $\lambda_{i}-\gamma\alpha_{i}=0$ for at least one $i\in{\cal I}$, we are able to represent $\zeta$ with at least one fewer vectors $\nabla g_{i}(x^{*})$. We may repeat this procedure until $\zeta=\sum_{j\in{\cal J}}\theta_{j}\nabla h_{j}(x^{*})$ for some $\{\theta_{j}:j\in{\cal J}\}$ not all zero. Then by the relation $\zeta\in-\partial^{\infty}\Phi(x^{*})$, it follows that

[TABLE]

Thus by Definition 1(b)(ii), $\{\nabla h_{j}(x^{*}):j\in{\cal J}\}$ must be linearly dependent. This contradicts the fact that $\{\nabla h_{j}(x^{*}):j\in{\cal J}\}$ is a basis. Hence BQ (16) holds.

(iii) When $g,h$ are smooth around $x^{*}$ and $\Omega$ is regular, it follows from (Rock98, , Theorem 6.14) that

[TABLE]

The proof for (iii) is exactly the same as that for (i) except that $\partial^{\infty}\Phi(x^{*})$ and (24) are replaced by $D^{*}\Psi(x^{*})(0)$ and (27), respectively.   $\square$

When the constraint region is so simple that its limiting normal cone is easy to calculate directly, we can use Proposition 3 to verify our proposed qualification conditions. The following simple minimax problem illustrates that conditions (16)–(17) hold and then $\partial^{\infty}$-quasi-normality, $\partial^{\infty}$-RCPLD, and $D^{*}$-quasi-normality are all satisfied since the constraint function is linear.

Example 3

Consider the following minimax problem

[TABLE]

Let $V(x):=\max\{-x^{3}+xy:y\geq 0\}$. Then it is easy to verify that the above minimax problem can be equivalently rewritten as

[TABLE]

where $V(x)=-x^{3}$ if $x\leq 0$ and $\infty$ otherwise. Clearly, $x^{*}=0$ is a minimizer of problem (28). We observe that $V$ is not continuous at $x^{*}$ but is lower semi-continuous at $x^{*}$. Moreover,

[TABLE]

which indicates that $-\partial^{\infty}V(x^{*})\cap{\cal N}_{\Re_{-}}(x^{*})=-D^{*}V(x^{*})(0)\cap{\cal N}_{\Re_{-}}(x^{*})=\{0\}$. Since the constraint function of problem (28) is linear, the standard RCPLD obviously holds and by (BertsekasJOTA2002, , Proposition 3.1), the standard quasi-normality is also satisfied. It then follows from Proposition 3 that $\partial^{\infty}$-quasi-normality, $\partial^{\infty}$-RCPLD, and $D^{*}$-quasi-normality are all satisfied at $x^{*}$ for problem (28).   $\square$

By using the outer semi-continuity of the horizon subdifferential mapping and Proposition 1, it is not difficult to show that both $\partial^{\infty}$-quasi-normality and $D^{*}$-quasi-normality are locally persistent as follows.

Proposition 4

If $\partial^{\infty}$-quasi-normality ($D^{*}$-quasi-normality) holds at $x^{*}\in{\cal F}$, then there exists $\delta_{0}>0$ such that $\partial^{\infty}$-quasi-normality ($D^{*}$-quasi-normality) holds at every point in ${\cal B}_{\delta_{0}}(x^{*})\cap{\cal F}$.

We now show that $\partial^{\infty}$-RCPLD is also locally persistent.

Proposition 5

If $\partial^{\infty}$-RCPLD holds at $x^{*}\in{\cal F}$, then there exists $\delta_{0}>0$ such that $\partial^{\infty}$-RCPLD holds at every point in ${\cal B}_{\delta_{0}}(x^{*})\cap{\cal F}$.

Proof

Assume that $\partial^{\infty}$-RCPLD holds at $x^{*}$. Let ${\cal J}\subseteq\{1,\ldots,m\}$ be such that $\{\nabla h_{j}(x^{*}):j\in{\cal J}\}$ is a basis for the ${\rm span}\,\{\nabla h_{j}(x^{*}):j=1,\ldots,m\}$. Then it is easy to see that there exists $\delta_{1}\in(0,\delta)$ such that $\{\nabla h_{i}(x):i\in{\cal I}\}$ is linearly independent for all $x\in{\cal B}_{\delta}(x^{*})$, where $\delta$ is given in Definition 1(b). Then it follows from Definition 1(b)(i) that $\{\nabla h_{j}(x):j\in{\cal J}\}$ is a basis for the ${\rm span}\,\{\nabla h_{j}(x):j=1,\ldots,m\}$ for any $x\in{\cal B}_{\delta_{1}}(x^{*})$. Let $\delta_{2}:=\delta_{1}/2$. Then by Definition 1(b)(i) again, $\{\nabla h_{j}(y):j=1,\ldots,m\}$ has the same rank for all $y\in{\cal B}_{\delta_{2}}(x)$ and $x\in{\cal B}_{\delta_{2}}(x^{*})$. Thus it suffices to show that there exists $\delta_{0}\in(0,\delta_{2})$ such that for any $x\in{\cal B}_{\delta_{0}}(x^{*})$, Definition 1(b)(ii) holds at $x$. Assume to the contrary that this is not true. That is, there exist a sequence $\{x^{k}\}$ converging to $x^{*}$, and $\{\lambda^{k}_{i}\geq 0:i\in{\cal I}^{k}\}$ with ${\cal I}^{k}\subseteq{\cal I}_{g}(x^{k})$ and $\{\mu^{k}_{j}:j\in{\cal J}\}$ not all zero such that

[TABLE]

and there exists a sequence $\{y^{k,l}\}_{l}$ converging to $x^{k}$ such that

[TABLE]

By the diagonalization law, there exists a sequence $\{z^{k}\}$ converging to $x^{*}$ such that

[TABLE]

Since $g$ is continuous, it is easy to verify that ${\cal I}_{g}(x^{k})\subseteq{\cal I}_{g}(x^{*})$ for any $k$ sufficiently large and hence ${\cal I}^{k}\subseteq{\cal I}_{g}(x^{*})$. Since the number of the possible sets ${\cal I}^{k}$ is finite, without loss of generality we may assume that ${\cal I}^{k}\equiv{\cal I}$ for any $k$ sufficiently large. Let

[TABLE]

Clearly, $t_{k}>0$ for any $k$. Without loss of generality, we may assume that

[TABLE]

It is easy to see that

[TABLE]

By the outer semi-continuity of the horizon subdifferential mapping, dividing (29) by $t_{k}$ and taking limits on both sides as $k\to\infty$ imply that

[TABLE]

The last two relations and $\partial^{\infty}$-RCPLD imply that for any $x\in{\cal B}_{\delta}(x^{*})$,

[TABLE]

which contradicts (30). The desired result follows immediately.   $\square$

4 Necessary optimality conditions

The purpose of this section is to show that the KKT condition defined below is necessary for optimality under $\partial^{\infty}$-quasi-normality or $\partial^{\infty}$-RCPLD.

Definition 3 (KKT condition)

Let $x^{*}\in{\cal F}$. We say that $x^{*}$ is a KKT point of problem (1) if there exist multipliers $\lambda\in\Re^{n}$ and $\mu\in\Re^{m}$ such that

[TABLE]

We first show that the KKT condition holds at a local minimizer under a weaker qualification condition.

Lemma 1

Let $x^{*}$ be a local minimizer of problem (1). Suppose that BQ (16) holds at $x^{*}$ and

[TABLE]

where ${\cal I}^{*}:={\cal I}_{g}(x^{*})$. Then $x^{*}$ is a KKT point.

Proof

It is clear that $x^{*}$ is a local minimizer of the problem

[TABLE]

Then by Fermat’s rule (see, e.g., (Rock98, , Theorem 10.1)), we have

[TABLE]

Since BQ (16) holds at $x^{*}$, it then follows from the sum rule for the limiting subdifferentials (see, e.g., (Rock98, , Corollary 10.9)) and the relation $\partial\delta_{\cal F}(x^{*})={\cal N}_{\cal F}(x^{*})$ that

[TABLE]

This and (31) imply the desired result immediately.   $\square$

The following result follows immediately from the fact that (31) may be implied by the local error bound condition (e.g., (int:loffe, , Proposition 3.4)).

Corollary 1

Let $x^{*}$ be a local minimizer of problem (1). If BQ (16) holds at $x^{*}$ and ${\cal F}$ admits a local error bound at $x^{*}$, then $x^{*}$ is a KKT point.

Let us revisit Example 1 which is in a four dimensional space. Even in this low dimensional space, it is not easy to calculate the limiting normal cone of the constraint region and hence BQ (16) is difficult to verify. For a constraint region involving many nonlinear constraints in high-dimensional spaces, it is almost impossible to calculate directly the limiting normal cone and thus BQ (16) is very difficult to verify. Fortunately, $\partial^{\infty}$-quasi-normality and $\partial^{\infty}$-RCPLD are expressed in terms of the problem data explicitly and hence much easier to verify. The following result shows that these two proposed qualification conditions are sufficient for the KKT condition to hold at a local minimizer.

Theorem 4.1

Let $x^{*}$ be a local minimizer of problem (1). Assume that either $\partial^{\infty}$-quasi-normality holds at $x^{*}$ or $\partial^{\infty}$-RCPLD holds at $x^{*}$ and $g,h$ are smooth around $x^{*}$. Then $x^{*}$ is a KKT point.

Proof

By Proposition 3, either the standard quasi-normality and BQ (16) or the standard RCPLD and BQ (16) hold. It then follows from (ye-zhang, , Proposition 4) and (Guo14, , Theorem 3.2) that condition (31) holds. Thus, the desired result follows from Lemma 1 immediately.   $\square$

Corollary 2

Let $x^{*}$ be a local minimizer of problem (1) and let $g,h$ be linear. Suppose also that the following implication holds:

[TABLE]

Then $x^{*}$ is a KKT point.

Proof

We first show that $\partial^{\infty}$-RCPLD holds at $x^{*}$. Since $h$ is linear, Definition 1(b)(i) holds. It then suffices to show that Definition 1(b)(ii) holds. Let ${\cal J}$ be such that $\{\nabla h_{j}(x^{*}):j\in{\cal J}\}$ is a basis for the ${\rm span}\,\{\nabla h_{j}(x^{*}):j=1,\ldots,m\}$ and ${\cal I}\subseteq{\cal I}^{*}$. Assume that there exist $\{\lambda_{i}:i\in{\cal I}\}$ and $\{\mu_{j}:j\in{\cal J}\}$ not all zero such that

[TABLE]

which implies that $\sum\limits_{i\in{\cal I}}\lambda_{i}\nabla g_{i}(x^{*})+\sum\limits_{j\in{\cal J}}\mu_{j}\nabla h_{j}(x^{*})=0$ by (32). This means that the family of gradients $\{\nabla g_{i}(x),\nabla h_{j}(x):i\in{\cal I},j\in{\cal J}\}$ is linearly dependent for all $x$ since $g,h$ are linear. Thus, $\partial^{\infty}$-RCPLD holds at $x^{*}$ and then the desired result follows immediately from Theorem 4.1.   $\square$

The following example illustrates the applicability of Corollary 2.

Example 4

Consider the following problem

[TABLE]

at a minimizer $x^{*}={(1,0)}$. By simple calculation, we have $\partial^{\infty}f(x^{*})=\{0\}\times\Re$ and then

[TABLE]

implies that $\lambda_{1}=\lambda_{2}$. Thus -\lambda_{1}\left(\begin{array}[]{c}1\\ 1\end{array}\right)+\lambda_{2}\left(\begin{array}[]{c}1\\ 1\end{array}\right)=\left(\begin{array}[]{c}0\\ 0\end{array}\right) and by Corollary 2, it then follows that $x^{*}$ is a KKT point.   $\square$

Letting $\Phi$ be an indicator function of a closed subset $\Omega$ in $\Re^{d}$, i.e., $\Phi(x)=\delta_{\Omega}(x)$, the following result follows immediately from Theorem 1, which extends the result of (RCPLD, , Corollary 1) to allow an extra abstract set constraint $x\in\Omega$ and the nonsmoothness of the objective function.

Corollary 3

Let $x^{*}$ be a local minimizer of the nonlinear program

[TABLE]

where $f,g,h$ are defined as in problem (1) and $\Omega$ is a closed subset in $\Re^{d}$. Here we assume that $g,h$ are smooth around $x^{*}$. Suppose further that RCPLD holds at $x^{*}$, i.e., there exists $\delta>0$ such that

• (i)

$\{\nabla h_{j}(x):j=1,\ldots,m\}$* has the same rank for each $x\in{\cal B}_{\delta}(x^{*})$;*

• (ii)

for each ${\cal I}\subseteq{\cal I}_{g}(x^{*})$, if there exist $\{\lambda_{i}\geq 0:i\in{\cal I}\}$ and $\{\mu_{j}:j\in{\cal J}\}$ not all zero such that

[TABLE]

then $\{\nabla g_{i}(x),\nabla h_{j}(x):i\in{\cal I},j\in{\cal J}\}$ is linearly dependent for each $x\in{\cal B}_{\delta}(x^{*})$,

where ${\cal J}\subseteq\{1,\ldots,m\}$ is such that $\{\nabla h_{j}(x):j\in{\cal J}\}$ is a basis for the ${\rm span}\,\{\nabla h_{j}(x):j=1,\ldots,m\}$. Then there exist multipliers $\lambda\in\Re^{n}$ and $\mu\in\Re^{m}$ such that

[TABLE]

5 Exact penalization

This section focuses on exact penalization for problem (1). We first give an exact penalization result for a special case of problem (1) where $\Phi$ is the sum of a composite function of a separable lower semi-continuous function with a continuous function and an indictor function of a closed subset. To this end, we give a characterization of the regular subdifferential as follows. It can be shown easily by using the definition of the regular subdifferential and thus we omit the proof here.

Lemma 2

Let $\psi:\Re^{d}\to(-\infty,\infty]$ be lower semi-continuous and $x^{*}\in\Re^{d}$ be such that $\psi(x^{*})$ is finite. Then $\hat{\partial}\psi(x^{*})=\Re^{d}$ if and only if for any $M>0$, there exists $\delta>0$ such that

[TABLE]

We are now ready to give the first main result on exact penalization.

Theorem 5.1

Assume that $x^{*}$ is a local minimizer of problem (1) where

[TABLE]

Here $\Omega$ is a closed subset in $\Re^{d}$ and for any $i=1,\ldots,s$, $\phi_{i}:\Re\to\Re$ is lower semi-continuous and $\omega_{i}(x):\Re^{d}\to\Re$ is continuous. Let $t^{*}:=\omega(x^{*})$, ${\cal I}:=\{i:\partial^{\infty}\phi_{i}(t_{i}^{*})=\{0\}\}$, and ${\cal I}^{c}$ be the complement of ${\cal I}$ with respect to $\{1,\ldots,s\}$. Assume further that $\hat{\partial}\phi_{i}(t^{*}_{i})=\Re$ for any $i\in{\cal I}^{c}$ and the following restricted system with respect to $(x,t)$:

[TABLE]

admits a local error bound at $(x^{*},t^{*})$. Then there exists $\rho_{0}>0$ such that for any $\rho\geq\rho_{0}$, $x^{*}$ is also a local minimizer of the exact penalization problem

[TABLE]

Proof

Since $x^{*}$ is a local minimizer of problem (1), it is not difficult to see that $(x^{*},t^{*})$ is a local minimizer of the following problem:

[TABLE]

We observe that $\Pi$ is Lipschitz around $(x^{*},t^{*})$ and denote by $L_{\Pi}$ the Lipschitz constant. Then by Clarke’s exact penalization principle (c, , Proposition 2.4.3), there exists $\delta_{1}>0$ such that

[TABLE]

where ${\cal F}^{\prime}$ denotes the constraint region of problem (38). Since ${\cal F}^{\prime}$ admits a local error bound at $(x^{*},t^{*})$, there exist $\delta_{2}\in(0,\delta_{1})$ and $\kappa>0$ such that for any $(x,t)\in{\cal B}_{\delta_{2}}(x^{*},t^{*})\cap(\Omega\times\Re^{s})$,

[TABLE]

For simplicity, the above local error bound is expressed under the $\ell_{1}$ norm. This together with (39) implies that for any $(x,t)\in{\cal B}_{\delta_{2}}(x^{*},t^{*})\cap(\Omega\times\Re^{s})$,

[TABLE]

Due to the continuity of function $w$, we may choose $\delta_{3}\in(0,\delta_{2})$ such that $(x,w(x))\in{\cal B}_{\delta_{2}}(x^{*},t^{*})$ for any $x\in{\cal B}_{\delta_{3}}(x^{*})$. Thus, by letting $t=w(x)$ in (40), it follows that for any $x\in{\cal B}_{\delta_{3}}(x^{*})\cap\Omega$,

[TABLE]

Since $\hat{\partial}\phi_{i}(t_{i}^{*})=\Re$ for any $i\in{\cal I}^{c}$, it then follows from Lemma 2 and the continuity of $w$ that there exists $\delta\in(0,\delta_{3})$ such that

[TABLE]

This and (41) imply that for any $x\in{\cal B}_{\delta}(x^{*})\cap\Omega$,

[TABLE]

Then the desired result follows immediately by the equivalence of all norms in finite dimensional spaces.   $\square$

It should be noted that Theorem 5.1 can be applied to a class of sparse optimization problems. For the widely used bridge penalty $\phi(t)=|t|^{p}$ with $p\in(0,1)$ in the sparse optimization literature, it is easy to see that $\phi$ is not Lipschitz around $t^{*}=0$. However, it is not hard to verify that $\hat{\partial}\phi(t^{*})=\Re$ and thus this bridge penalty function is a suitable outer function required in Theorem 5.1. In the following, we give some exact penalization results for problem (1) where the objective function is related to the bridge penalty function.

The following result shows that the problem considered in Chen13 with an extra abstract constraint set which is the union of finitely many polyhedral sets admits an exact penalization.

Corollary 4

Assume that $x^{*}$ is a local minimizer of problem (1) where

[TABLE]

with $a_{i}\in\Re^{d}$, $p\in(0,1)$, and $\Omega\subseteq\Re^{d}$ which is the union of finitely many polyhedral sets. Assume further that $g,h$ are linear. Then there exists $\rho_{0}>0$ such that for any $\rho\geq\rho_{0}$, $x^{*}$ is also a local minimizer of the exact penalization problem

[TABLE]

Proof

Let $\phi(t):=|t|^{p}$ and $t_{i}^{*}:=a_{i}^{T}x^{*}\ i=1,\ldots,s$. It is easy to verify that

[TABLE]

and $\hat{\partial}\phi(t_{i}^{*})=\Re$ for any $i\in{\cal I}^{c}$ where ${\cal I}^{c}$ is the complement of ${\cal I}$ with respect to $\{1,\ldots,s\}$. Since the constraint set

[TABLE]

is the union of finitely many polyhedral sets, it then follows from the corollary in (Robinson81, , Page 210) that the local error bound condition holds everywhere for the constraint set

[TABLE]

Then the desired result follows immediately from Theorem 5.1.   $\square$

We next give an exact penalization result for a problem which is more general than the one considered in YLiu .

Corollary 5

Assume that $x^{*}$ is a local minimizer of problem (1) where

[TABLE]

with $a_{i}\in\Re^{d},b_{i}\in\Re$, $p\in(0,1)$, and $\Omega\subseteq\Re^{d}$ which is the union of finitely many polyhedral sets. Assume further that $g,h$ are linear. Then there exists $\rho_{0}>0$ such that for any $\rho\geq\rho_{0}$, $x^{*}$ is also a local minimizer of the exact penalization problem

[TABLE]

Proof

Let $\phi(t):=|t|^{p}$ and $t_{i}^{*}:=(b_{i}-a_{i}^{T}x^{*})_{+}\ i=1,\ldots,s$. Using the same notations ${\cal I},{\cal I}^{c}$ as in the proof of Corollary 4, it suffices to investigate the local error bound condition for the constraint set

[TABLE]

It is easy to see that the parametric counterpart of the above set

[TABLE]

is the union of finitely many polyhedral sets. Thus by the corollary in (Robinson81, , Page 210), the desired local error bound condition is satisfied. The proof is complete by applying Theorem 5.1.   $\square$

In the rest of this section, we investigate sufficient conditions ensuring an exact penalization for problem (1) where $\Phi$ is the sum of a continuous function and an indictor function of a closed subset. In particular, we investigate exact penalization for the following problem:

[TABLE]

where $f,g,h$ are defined as in problem (1), $\Psi:\Re^{d}\to\Re$ is a continuous function, and $\Omega$ is a closed subset in $\Re^{d}$. As discussed in Section 1, when the objective function of a nonlinear program is locally Lipschitz, the admittance of the local error bound for its constraint region is sufficient to ensure an exact penalization. For this purpose, we introduce the following auxiliary problem where the objective function is locally Lipschitz:

[TABLE]

In the case where $\Omega=\Re^{d}$, the constraint region of problem (5) can be rewritten as

[TABLE]

We observe that by using the definition of the coderivative, the inclusion

[TABLE]

can be rewritten as

[TABLE]

Hence if $x^{*}$ is $D^{*}$-quasinormal for problem (5), then $(x^{*},\Psi(x^{*}))$ is quasinormal for problem (5). Moreover, ${\rm gph}\,\Psi$ is a closed subset in $\Re^{d+1}$ by the continuity of $\Psi$. These and the local Lipschitzness of $g,h$ enable us to apply (guoyezhang-infinite, , Corollary 5.3) to derive the local error bound condition at $(x^{*},\Psi(x^{*}))$, that is, there exist $\delta>0$ and $\kappa>0$ such that

[TABLE]

The exact penalization result then follows from applying Clarke’s exact penalization principle (c, , Proposition 2.4.3) to problem (5). Unfortunately, the above argument does not work for the case where the abstract constraint set $\Omega$ is not equal to the whole space $\Re^{d}$. Nevertheless, we have succeeded in deriving the following local error bound result under $D^{*}$-quasi-normality given in Definition 2.

Lemma 3

Suppose that $D^{*}$-quasi-normality holds at $x^{*}\in{\cal F}$. Then the set

[TABLE]

admits a local error bound at $(x^{*},y^{*})$ with $y^{*}:=\Psi(x^{*})$, that is, there exist $\delta>0$ and $\kappa>0$ such that

[TABLE]

Proof

First we observe that $\Lambda$ defined in (44) can be rewritten as

[TABLE]

where

[TABLE]

with $H(x,y):=|G(x,y)|$ and $G(x,y):=\Psi(x)-y$. Then in order to obtain the desired result, by (error02, , Theorem 3.1), it suffices to show that there exist $\tilde{\delta}>0$ and $\tilde{\kappa}>0$ such that

[TABLE]

We now make some preparations for subsequent analysis. Since $|\cdot|$ is globally Lipschitz and $G$ is continuous, it follows from Proposition 2(iii) that for any $(x,y)$,

[TABLE]

Since ${\rm dist}_{\Omega}(\cdot)$ is globally Lipschitz and $\Xi$ is continuous, by Proposition 2(i), we have that for any $(x,y)$,

[TABLE]

Since $g,h$ are both Lipschitz around $x^{*}$, by Proposition 2(iv), it follows that for any $(x,y)$ with $x$ sufficiently close to $x^{*}$, there exists $(\alpha,\beta,\gamma)\in{\cal M}(x,y)$ where

[TABLE]

such that

[TABLE]

In the following, we prove (45) by contradiction. Assume to the contrary that there exist a sequence $\{(x^{k},y^{k})\}$ with $(x^{k},y^{k})\in\Omega\times\Re\backslash\Lambda$ converging to $(x^{*},y^{*})$ and $\pi^{k}\in\partial(\Xi+{\mathop{\rm dist}}_{\Omega})(x^{k},y^{k})$ such that $\pi^{k}\to 0$. Then it follows from (52)–(54) that there exists $(\alpha^{k},\beta^{k},\gamma^{k})\in{\cal M}(x^{k},y^{k})$ such that

[TABLE]

Noting that $(x^{k},y^{k})\in\Omega\times\Re\backslash\Lambda$, we have that

[TABLE]

Since $(\alpha^{k},\beta^{k},\gamma^{k})\in{\cal M}(x^{k},y^{k})$, it follows that $\alpha^{k}\geq 0$, $\beta^{k}\geq 0,\ \gamma^{k}\geq 0$, and

[TABLE]

Define

[TABLE]

Since it follows from (56) and (60) that $\gamma_{j}^{k}=0$ when $h_{j}(x^{k})=0$, it is easy to see that $\|\gamma^{k}\|_{1}=\|\bar{\gamma}^{k}\|_{1}$. It then follows from (57) that

[TABLE]

Moreover, by Proposition 2(ii), we have that

[TABLE]

We continue the proof by considering the two separate cases as follows.

Case (a): There exists a subsequence $\{\alpha^{k}\}_{k\in{\cal K}}$ with ${\cal K}\subseteq{\mathbb{N}}$ such that $\alpha^{k}=0$ for any $k\in{\cal K}$. Then it follows from (51), (55), (62), and the definition of notation $\diamond$ that for any $k\in{\cal K}$,

[TABLE]

In this case, it follows from (61) that $\|\beta^{k}\|_{1}+\|\bar{\gamma}^{k}\|_{1}=1$. Thus, there must exist subsequences $\{\beta^{k}\}_{k\in{\cal K}_{1}}$ and $\{\bar{\gamma}^{k}\}_{k\in{\cal K}_{1}}$ with ${\cal K}_{1}\subseteq{\cal K}$ such that as ${\cal K}_{1}\ni k\to\infty$,

[TABLE]

Taking limits on both sides of (63), it then follows from (64), Proposition 1, and the local boundedness of the limiting subdifferential of local Lipschitz functions that

[TABLE]

If $g_{i}(x^{*})<0$, then $g_{i}(x^{k})<0$ for any $k$ sufficiently large. Thus by (56) and (59), $\beta_{i}^{k}=0$ for any $k$ sufficiently large. This together with (64) implies that $\beta^{*}_{i}=0$. In conclusion, we have

[TABLE]

Moreover, if $\beta_{i}^{*}>0$, then by (64), we have $\beta^{k}_{i}>0$ for any $k\in{\cal K}_{1}$ sufficiently large. This and (59) imply that $g_{i}(x^{k})=\Xi(x^{k},y^{k})>0$. If $\gamma_{j}^{*}\neq 0$, then by (64), we have $\gamma_{j}^{*}\bar{\gamma}^{k}_{j}>0$ for any $k\in{\cal K}_{1}$ sufficiently large. Thus, it follows from the definition of $\bar{\gamma}^{k}_{j}$ and the relation $\gamma^{k}_{j}\geq 0$ that $\gamma_{j}^{*}h_{j}(x^{k})>0$ for any $k\in{\cal K}_{1}$ sufficiently large. Thus, we have

[TABLE]

which together with (64)–(66) contradicts $D^{*}$-quasi-normality at $x^{*}$ by using the relation $\partial{\rm dist}_{\Omega}(x^{*})\subseteq{\cal N}_{\Omega}(x^{*})$.

Case (b): There exists a subsequence $\{\alpha^{k}\}_{k\in{\cal K}}$ with ${\cal K}\subseteq{{\mathbb{N}}}$ such that $\alpha^{k}>0$ for any $k\in{\cal K}$. In this case, it then follows from (48), (55), (62), and the definition of notation $\diamond$ that for any $k\in{\cal K}$, there exists $\xi^{k}\in\partial|G(x^{k},y^{k})|$ such that

[TABLE]

Since $\alpha^{k}>0$, it follows from (56) and (58) that $H(x^{k},y^{k})=\Xi(x^{k},y^{k})>0$. Thus, by direct calculation, we have that

[TABLE]

It then follows that $|\bar{\alpha}^{k}|=\alpha^{k}$ where $\bar{\alpha}^{k}:=\alpha^{k}\xi^{k}$. Thus by (61), it follows that

[TABLE]

Without loss of generality, we assume that as ${\cal K}\ni k\to\infty$,

[TABLE]

It then follows from (67) and the relation $\pi^{k}\to 0$ that $\bar{\alpha}^{k}\to 0$ as ${\cal K}\ni k\to\infty$. Thus by (71), we have that

[TABLE]

Taking limits on both sides of (67), it then follows from (71), Proposition 1, and the local boundedness of the limiting subdifferential of local Lipschitz functions that

[TABLE]

The rest of the proof for case (b) is similar to that for case (a).

Therefore, there exist $\tilde{\delta}>0$ and $\tilde{\kappa}>0$ such that (45) holds and thus by (error02, , Theorem 3.1), we obtain the desired result immediately.   $\square$

We are now ready to give the exact penalization result for problem (5).

Theorem 5.2

Let $x^{*}$ be a local minimizer of problem (5). If $D^{*}$-quasi-normality holds at $x^{*}$, then there exists $\rho_{0}>0$ such that for any $\rho\geq\rho_{0}$, $x^{*}$ is also a local minimizer of the exact penalization problem

[TABLE]

Proof

By the local optimality of $x^{*}$, it is easy to see that $(x^{*},y^{*})$ with $y^{*}:=\Psi(x^{*})$ is a local minimizer of the following auxiliary problem:

[TABLE]

where $\Lambda$ is defined in Lemma 3. Denote by $\ell$ the Lipschitz constant of the objective function $f(x)+y$ around $(x^{*},y^{*})$. By Clarke’s exact penalization principle (c, , Proposition 2.4.3), there exists $\delta_{1}>0$ such that

[TABLE]

Then it follows from Lemma 3 that there exist $\delta_{2}\in(0,\delta_{1})$ and $\kappa>0$ such that for all $(x,y)\in{\cal B}_{\delta_{2}}(x^{*},y^{*})\cap(\Omega\times\Re)$,

[TABLE]

By the continuity of $\Psi$, we may choose $\delta\in(0,\delta_{2})$ such that $(x,\Psi(x))\in{\cal B}_{\delta_{2}}(x^{*},y^{*})$ for any $x\in{\cal B}_{\delta}(x^{*})$. Then the desired result follows immediately from (73) by letting $y=\Psi(x)$ and $\rho_{0}:=\kappa\ell$.   $\square$

Acknowledgements.

We thank the referees for their helpful suggestions and comments that have helped us to improve the presentation of the paper. We would also like to thank Jim Burke for a discussion on the topic of this research.