Extended Trust-Region Problems with One or Two Balls: Exact Copositive and Lagrangian Relaxations

TL;DR

This paper introduces geometric conditions under which copositive relaxations provide exact solutions for certain nonconvex quadratic problems, surpassing traditional Lagrangian relaxations, with practical verifiability via linear optimization.

Contribution

It establishes new conditions for exact copositive relaxation in extended trust-region problems, improving solution accuracy over standard methods.

Findings

Copositive relaxation is tighter than Lagrangian relaxation.

Certain quadratic problems have exact copositive relaxation with infinite Lagrangian duality gap.

Verifiable conditions for exact relaxations via linear optimization.

Abstract

We establish a geometric condition guaranteeing exact copositive relaxation for the nonconvex quadratic optimization problem under two quadratic and several linear constraints, and present sufficient conditions for global optimality in terms of generalized Karush-Kuhn-Tucker multipliers. The copositive relaxation is tighter than the usual Lagrangian relaxation. We illustrate this by providing a whole class of quadratic optimization problems that enjoys exactness of copositive relaxation while the usual Lagrangian duality gap is infinite. Finally, we also provide verifiable conditions under which both the usual Lagrangian relaxation and the copositive relaxation are exact for an extended CDT (two-ball trust-region) problem. Importantly, the sufficient conditions can be verified by solving linear optimization problems.

Equations

$$(\mathrm{P})$$

$${\mathsf{A}}\oplus{\mathsf{B}}=\left[\begin{array}[]{cc}{\mathsf{A}}&{\mathsf{O}}\\ {\mathsf{O}}^{\top}&{\mathsf{B}}\end{array}\right]\,,\quad\mbox{and in particular we will use}\quad{\mathsf{J}}_{0}:=1\oplus{\mathsf{O}}=\left[\begin{array}[]{cc}1&{\mathsf{o}}^{\top}\\ {\mathsf{o}}&{\mathsf{O}}\end{array}\right]\,.$$

$${\mathsf{M}}(q):=\left[\begin{array}[]{cc}\gamma&-{\mathsf{d}}^{\top}\\ -{\mathsf{d}}&{\mathsf{H}}\end{array}\right]\,.$$

$${\mathcal{C}}^{\star }:=\left\{\mathsf{S}={\mathsf{S}}^{\top }\in {\mathbb{R}}^{n\times n}:⟨\mathsf{S},\mathsf{X}⟩\ge 0\text{mbox}forall\mathsf{X}\in \mathcal{C}\right\}$$

$$\begin{array}[]{rl}{\mathsf{Q}}\mbox{ is }\Gamma\mbox{-copositive if }&{\mathsf{v}}^{\top}{\mathsf{Q}}{\mathsf{v}}\geq 0\mbox{ for all }{\mathsf{v}}\in\Gamma\,,\quad\mbox{and that }\\[3.00003pt] {\mathsf{Q}}\mbox{ is strictly }\Gamma\mbox{-copositive if }&{\mathsf{v}}^{\top}{\mathsf{Q}}{\mathsf{v}}>0\mbox{ for all }{\mathsf{v}}\in\Gamma\setminus\left\{{\mathsf{o}}\right\}\,.\end{array}$$

$${\mathcal{C}}_{\Gamma }^{\star }:=\left\{\mathsf{Q}={\mathsf{Q}}^{\top }\in {\mathbb{R}}^{n\times n}:\mathsf{Q}\text{mbox}is\Gamma \text{mbox}-copositive\right\}$$

$${\mathcal{C}}_{\Gamma }:=\left\{\mathsf{X}=\mathsf{F}{\mathsf{F}}^{\top }:\mathsf{F}\text{mbox}has\left(\genfrac{}{}{0ex}{}{n+1}{2}\right)\text{mbox}columnsin\Gamma \right\},$$

$${\mathcal{C}}_{\Gamma }=\text{mbox}conv\left\{\mathsf{x}{\mathsf{x}}^{\top }:\mathsf{x}\in \Gamma \right\},$$

$${\mathsf{M}}=\left[\begin{array}[]{rl}{\mathsf{R}}&{\mathsf{S}}^{\top}\\ {\mathsf{S}}&{\mathsf{H}}\end{array}\right]\quad\mbox{where}\quad{\mathsf{R}}\mbox{ is a }p\times p\mbox{-matrix.}$$

$$(\mathrm{P})$$

$$z^{\ast }:=\inf \left\{f_0(\mathsf{x}):\mathsf{x}\in F\cap P\right\}\text{mbox}withP:=\left\{\mathsf{x}\in {\mathbb{R}}^n:\mathsf{B}\mathsf{x}\le \mathsf{b}\right\},$$

$$\bar{P}:=\left\{\mathsf{y}=(\mathsf{s},\mathsf{x})\in {\mathbb{R}}_{+}^p\times {\mathbb{R}}^n:\bar{\mathsf{B}}\mathsf{y}=\mathsf{b}\right\}$$

$$z^{\ast }=\inf \left\{{\bar{f}}_0(\mathsf{y}):{\bar{f}}_i(\mathsf{y})\le 0,i\in [1:3],\mathsf{y}=(\mathsf{s},\mathsf{x})\in {\mathbb{R}}_{+}^p\times {\mathbb{R}}^n\right\}.$$

$$L(\mathsf{y};\mathsf{u},\mathsf{v}):={\bar{f}}_0(\mathsf{y})+u_1{\bar{f}}_1(\mathsf{y})+u_2{\bar{f}}_2(\mathsf{y})+u_3{\bar{f}}_3(\mathsf{y})-{\mathsf{v}}^{\top }\mathsf{s}.$$

$$\left[\begin{array}[]{ccc}-u_{1}-u_{2}+u_{3}{\|{\mathsf{b}}\|}^{2}-\mu&-u_{3}{\mathsf{b}}^{\top}&{\mathsf{q}}_{0}^{\top}+u_{1}{\mathsf{q}}_{1}^{\top}-u_{2}{\mathsf{a}}^{\top}{\mathsf{A}}-u_{3}{\mathsf{b}}^{\top}{\mathsf{B}}\\ -u_{3}{\mathsf{b}}&u_{3}{\mathsf{I}}_{p}&u_{3}{\mathsf{B}}\\ {\mathsf{q}}_{0}+u_{1}{\mathsf{q}}_{1}-u_{2}{\mathsf{A}}^{\top}{\mathsf{a}}-u_{3}{\mathsf{B}}^{\top}{\mathsf{b}}&u_{3}{\mathsf{B}}^{\top}&{\mathsf{H}}_{u}+u_{3}{\mathsf{B}}^{\top}{\mathsf{B}}\end{array}\right]$$

$$(\mathrm{COP})z_{\mathrm{COP}}^{\ast }:=\sup \left\{\mu :(\mu ,\mathsf{u})\in \mathbb{R}\times {\mathbb{R}}_{+}^3,\mathsf{M}(L(\cdot ;\mathsf{u},\mathsf{o}))-\mu {\mathsf{J}}_0\text{mbox}is{\rm Y}\text{mbox}-copositive\right\},$$

$$z_{\mathrm{LD}}^{\ast }:=\sup \left\{\Theta (\mathsf{u},\mathsf{v}):(\mathsf{u},\mathsf{v})\in {\mathbb{R}}_{+}^3\times {\mathbb{R}}_{+}^p\right\},$$

$$z_{\mathrm{semi}}^{\ast }:=\sup \left\{{\Theta }_{\mathrm{semi}}(\mathsf{u}):\mathsf{u}\in {\mathbb{R}}_{+}^3\right\}.$$

$$z_{\mathrm{LD}}^{\ast }\le z_{\mathrm{COP}}^{\ast }=z_{\mathrm{semi}}^{\ast }\le z^{\ast }.$$

$$\min$$

$$\underset{(\mathsf{x},\mathsf{s})\in {\mathbb{R}}^2\times {\mathbb{R}}^2}{\min }$$

$$z_{\rm COP}^{*}=\sup_{\mu\in{\mathbb{R}},u\geq 0}\left\{\mu:\left[\begin{array}[]{ccccc}-\mu&0&0&0&0\\ 0&u&0&-u&0\\ 0&0&u&0&-u\\ 0&-u&0&\frac{1}{2}+u&1\\ 0&0&-u&1&1+u\end{array}\right]\;\mbox{ is }(\mathbb{R}^{3}_{+}\times\mathbb{R}^{2})\mbox{-copositive}\right\}$$

$$\left[\begin{array}[]{ccccc}0&0&0&0&0\\ 0&1&0&-1&0\\ 0&0&1&0&-1\\ 0&-1&0&\frac{3}{2}&1\\ 0&0&-1&1&2\end{array}\right]\;\mbox{ is }(\mathbb{R}^{3}_{+}\times\mathbb{R}^{2})\mbox{-copositive.}$$

$$\underset{(\mathsf{x},\mathsf{s})\in {\mathbb{R}}^2\times {\mathbb{R}}^2}{\inf }\{[\frac{1}{2}{x}_1^2+2x_1{x}_2+x_2^2]+u[(-x_1+s_1{)}^2+(-x_2+s_1{)}^2]+v_1{s}_1+v_2{s}_2\}=-\infty .$$

$$[\frac{1}{2}{x}_1^2+2x_1{x}_2+x_2^2]+u[(-x_1+s_1{)}^2+(-x_2+s_1{)}^2]+v_1{s}_1+v_2{s}_2$$

$$L(\mathsf{y};\mathsf{u},\mathsf{v})={\bar{f}}_0(\mathsf{y})+\sum _{i=1}^3{u}_i{\bar{f}}_i(\mathsf{y})-{\mathsf{v}}^{\top }\mathsf{s},(\mathsf{u},\mathsf{v})\in {\mathbb{R}}_{+}^3\times {\mathbb{R}}^p.$$

$$\Omega :=\{[{\bar{f}}_0(\mathsf{y}),{\bar{f}}_1(\mathsf{y}),{\bar{f}}_2(\mathsf{y}),{\bar{f}}_3(\mathsf{y}){]}^{\top }:\mathsf{y}\in {\mathbb{R}}_{+}^p\times {\mathbb{R}}^n\}+{\mathbb{R}}_{+}^4.$$

$$\inf \left\{q(\mathsf{y}):\mathsf{y}\in {\mathbb{R}}_{+}^p\times {\mathbb{R}}^n\right\}=\sup \left\{\mu \in \mathbb{R}:\mathsf{M}(q)-\mu {\mathsf{J}}_0\text{mbox}is{\rm Y}\text{mbox}-copositive\right\}.$$

$${\Theta }_{\mathrm{semi}}(\mathsf{u})=\sup \left\{\mu :\mu \in \mathbb{R},\mathsf{M}(L(\cdot ;\mathsf{u},\mathsf{o}))-\mu {\mathsf{J}}_0\text{mbox}is{\rm Y}\text{mbox}-copositive\right\}$$

$$z_{\mathrm{semi}}^{\ast }$$

Full text

Extended Trust–Region Problems with One or Two Balls:

Exact Copositive and Lagrangian Relaxations

Immanuel M. Bomze

ISOR and VCOR, University of Vienna, Austria

Vaithilingam Jeyakumar

School of Mathematics and Statistics, University of New South Wales, Sydney, Australia

Guoyin Li

School of Mathematics and Statistics, University of New South Wales, Sydney, Australia

We establish a geometric condition guaranteeing exact copositive relaxation for the nonconvex quadratic optimization problem under two quadratic and several linear constraints, and present sufficient conditions for global optimality in terms of generalized Karush-Kuhn-Tucker multipliers. The copositive relaxation is tighter than the usual Lagrangian relaxation. We illustrate this by providing a whole class of quadratic optimization problems that enjoys exactness of copositive relaxation while the usual Lagrangian duality gap is infinite. Finally, we also provide verifiable conditions under which both the usual Lagrangian relaxation and the copositive relaxation are exact for an extended CDT (two-ball trust-region) problem. Importantly, the sufficient conditions can be verified by solving linear optimization problems.

Key words: Copositive matrices, non-convex optimization, quadratic optimization, quadratically constrained problem, global optimality condition, relaxation.

Revised Version: September 15, 2017

1 Introduction

Consider the following nonconvex quadratic optimization problem, which is referred to as the extended trust region problem:

[TABLE]

where ${\mathsf{Q}}_{0},{\mathsf{Q}}_{1}$ are $(n\times n)$ symmetric matrices, ${\mathsf{A}}$ is an $(\ell\times n)$ matrix, ${\mathsf{B}}$ is an $(m\times n)$ matrix, ${\mathsf{a}}\in\mathbb{R}^{\ell}$, ${\mathsf{b}}\in\mathbb{R}^{m}$ and ${\mathsf{q}}_{0},{\mathsf{q}}_{1}\in\mathbb{R}^{n}$. Model problems of this form arise from robust optimization problems under matrix norm or polyhedral data uncertainty [5, 20] and the application of the trust region method [15] for solving constrained optimization problems, such as nonlinear optimization problems with nonlinear and linear inequality constraints [9, 34]. It covers many important and challenging quadratic optimization (QP) problems such as those with box constraints; trust region problems with additional linear constraints; and the CDT (Celis-Dennis-Tapia or two-ball trust-region) problem [1, 9, 14, 28, 38]. In general, with no further structure on the additional linear constraints ${\mathsf{B}}{\mathsf{x}}\leq{\mathsf{b}}$, the model problem (P) is NP-hard as it encompasses the quadratic optimization problem with box constraints.

In the special case where ${\mathsf{Q}}_{1}$ is the identity matrix, ${\mathsf{A}},{\mathsf{B}}$ are zero matrices and ${\mathsf{a}},{\mathsf{b}},{\mathsf{q}}_{1}$ are zero vectors, the model problem (P) reduces to the well-known trust-region model. It has been extensively studied from both theoretical and algorithmic points of view [19, 39]. The trust-region problem enjoys exact Lagrangian relaxations. Moreover, its solution can be found by solving a dual Lagrangian system or, equivalently, a semidefinite optimization problem (SDP). Unfortunately, these nice features do not continue to hold for the more general extended trust-region problem (P); see [20]. In fact, it has been shown that exactness of Lagrangian (or SDP) relaxation can fail for the CDT problem, or for the trust region problem with only one additional linear inequality constraint.

Recently, copositive optimization has emerged as one of the important tools for studying nonconvex quadratic optimization problems. Copositive optimization is a special case of convex conic optimization (namely, to minimize a linear function over a cone subject to linear constraints). By now, equivalent copositive reformulations for many important problems are known, among them (non-convex, mixed-binary, fractional) quadratic optimization problems under a mild assumption [2, 3, 13], and some special optimization problems under uncertainty [4, 18, 32, 37]. In particular, it has been shown in [7] that, for quadratic optimization problems with additional nonnegative constraints, copositive relaxations (and its tractable approximations) provides a tighter bound than the usual Lagrangian relaxation. On the other hand, the techniques in [7] are not directly applicable because our model problem does not require the variables to be nonnegative.

In light of rapid evolution of this field, in this paper, we introduce a new copositive relaxation for the extended trust region problem (P), and present two significant contributions to copositive optimization:

• •

We establish a geometric condition guaranteeing exact copositive relaxation for the nonconvex quadratic optimization problem (P). We also present sufficient conditions for global optimality in terms of generalized Karush-Kuhn-Tucker multipliers extending the global optimality conditions obtained for CDT problems [9]. Moreover, we provide a class of quadratic optimization problems that enjoys exactness of the copositive relaxation while the usual Lagrangian duals for these problems yield trivial lower bounds with infinite gaps.

• •

In the special case, where (P) is an extended CDT (or two-ball trust region, TTR) problems, we also derive simple verifiable sufficient conditions, which is independent of the geometric conditions, ensuring both exact copositive relaxation and exact Lagrangian relaxations. In particular, the sufficient conditions can be checked by solving a linear optimization problem.

The paper is organized as follows: In Section 2, we first recall notation and terminology, and present some basic facts on copositivity. In Section 3, we introduce the copositive relaxation for (P) and its semi-Lagrangian reformulation. We also provide a global optimality condition and prove an exactness result for this relaxation. In Section 4, we examine the extended CDT problem and provide simple conditions ensuring the tightness of both the copositive relaxation and the usual Lagrangian relaxation. In the appendix, we provide details on how copositive relaxation problems can be approximated by hierarchies of semidefinite and/or linear optimization problems.

2 Preliminaries

We abbreviate by $[m\!:\!n]:=\left\{m,m+1,\ldots,n\right\}$ the integer range between two integers $m,n$ with $m\leq n$. By bold-faced lower-case letters we denote vectors in $n$-dimensional Euclidean space ${\mathbb{R}}^{n}$, by bold-faced upper case letters matrices, and by ⊤ transposition. The positive orthant is denoted by ${\mathbb{R}}^{n}_{+}:=\left\{{\mathsf{x}}\in{\mathbb{R}}^{n}:x_{i}\geq 0\mbox{ for all }i{\in\![{1}\!:\!{n}]}\right\}$. ${\mathsf{I}}_{n}$ is the $n\times n$ identity matrix. The letters ${\mathsf{o}}$ and ${\mathsf{O}}$ stand for zero vectors, and zero matrices, respectively, of appropriate orders. The set of all $n\times n$ matrices is denoted by ${\mathbb{R}}^{n\times n}$, and the closure (resp. interior) of a set $S\subset{\mathbb{R}}^{n}$ by ${\rm cl}(S)$ (resp. ${\rm int}~{}S$).

For a given symmetric matrix ${\mathsf{H}}={\mathsf{H}}^{\top}$, we denote the fact that ${\mathsf{H}}$ is positive-semidefinite by ${\mathsf{H}}\succeq{\mathsf{O}}$. Sometimes we write instead ”${\mathsf{H}}$ is psd.” Denoting the smallest eigenvalue of any symmetric matrix ${\mathsf{M}}={\mathsf{M}}^{\top}$ by $\lambda_{\rm min}({\mathsf{M}})$, we thus have ${\mathsf{H}}\succeq{\mathsf{O}}$ if and only if $\lambda_{\rm min}({\mathsf{H}})\geq 0$. Linear forms in symmetric matrices ${\mathsf{X}}$ will play an important role in this paper; they are expressed by Frobenius duality $\langle{{\mathsf{S}}},{{\mathsf{X}}}\rangle=\mbox{trace}({\mathsf{S}}{\mathsf{X}})$, where ${\mathsf{S}}={\mathsf{S}}^{\top}$ is another symmetric matrix of the same order as ${\mathsf{X}}$. By ${\mathsf{A}}\oplus{\mathsf{B}}$ we denote the direct sum of two square matrices:

[TABLE]

For any optimization problem, say (Q), we denote by val(Q) its optimal objective value (attained or not). Consider a quadratic function $q({\mathsf{x}})={\mathsf{x}}^{\top}{\mathsf{H}}{\mathsf{x}}-2{\mathsf{d}}^{\top}{\mathsf{x}}+\gamma$ defined on ${\mathbb{R}}^{n}$, with $q({\mathsf{o}})=\gamma$, $\nabla q({\mathsf{o}})=-2{\mathsf{d}}$ and $D^{2}q({\mathsf{o}})=2{\mathsf{H}}$ (the factors $2$ being here just for ease of later notation). For this $q$ we define the Shor relaxation matrix [36] as

[TABLE]

Then $q({\mathsf{x}})\geq 0$ for all ${\mathsf{x}}\in{\mathbb{R}}^{n}$ if and only if ${\mathsf{M}}(q)\succeq{\mathsf{O}}$.

Given any cone ${\mathcal{C}}$ of symmetric $n\times n$ matrices,

[TABLE]

denotes the dual cone of ${\mathcal{C}}$. For instance, if ${\mathcal{C}}=\left\{{\mathsf{X}}={\mathsf{X}}^{\top}\in{\mathbb{R}}^{n\times n}:{\mathsf{X}}\succeq{\mathsf{O}}\right\}$, then ${\mathcal{C}}^{\star}={\mathcal{C}}$ itself, an example of a self-dual cone. Trusting the sharp eyes of our readers, we chose a notation with subtle differences between the five-star denoting a dual cone, e.g., ${\mathcal{C}}^{\star}$, and the six-star, e.g. $z^{*}$, denoting optimality.

The key notion used below is that of copositivity. Given a symmetric $n\times n$ matrix ${\mathsf{Q}}$, and a closed, convex cone $\Gamma\subseteq{\mathbb{R}}^{n}$, we say that

[TABLE]

Strict copositivity generalizes positive-definiteness (all eigenvalues strictly positive) and copositivity generalizes positive-semidefiniteness (no eigenvalue strictly negative) of a symmetric matrix. Checking copositivity is NP-hard for most cones $\Gamma$ of interest, see [16, 31] for the classical case $\Gamma={\mathbb{R}}^{n}_{+}$ studied already by Motzkin [30] who coined the notion back in 1952. In the sequel, we will use ”copositive” synonymous for ”${\mathbb{R}}^{n}_{+}$-copositive” in Motzkin’s sense.

The set of all $\Gamma$-copositive matrices forms a closed, convex matrix cone, the copositive cone

[TABLE]

with non-empty interior ${\rm int}~{}{\mathcal{C}}^{\star}_{\Gamma}$, which exactly consists of all strictly $\Gamma$-copositive matrices. However, the cone ${\mathcal{C}}^{\star}_{\Gamma}$ is not self-dual. Rather one can show that ${\mathcal{C}}^{\star}_{\Gamma}$ is the dual cone of

[TABLE]

the cone of $\Gamma$-completely positive (cp) matrices. Note that the factor matrix ${\mathsf{F}}$ has many more columns than rows. A perhaps more amenable representation is

[TABLE]

where ${\mbox{conv }}S$ stands for the convex hull of a set $S\subset{\mathbb{R}}^{n}$. Caratheodory’s theorem then elucidates the bound ${n+1\choose 2}$ on the number of columns in ${\mathsf{F}}$ above, which is not sharp in the classical case $\Gamma={\mathbb{R}}^{n}_{+}$ but asymptotically tight [10, 35].

Next, we specify a result on reducing $\Upsilon$-copositivity with $\Upsilon={\mathbb{R}}^{p}_{+}\times{\mathbb{R}}^{n}$ to a combination of psd and classical copositivity conditions. This result will be used later on.

Lemma 2.1

Let $\Upsilon={\mathbb{R}}^{p}_{+}\times{\mathbb{R}}^{n}$ and partition a $(p+n)\times(p+n)$ matrix ${\mathsf{M}}$ as follows:

[TABLE]

Then ${\mathsf{M}}$ is $\Upsilon$-copositive if and only if the following two conditions hold:

(a)

${\mathsf{H}}$* is positive semidefinite and ${\mathsf{H}}{\mathsf{H}}^{\dagger}{\mathsf{S}}={\mathsf{S}}$, i.e., $\mbox{ker }{\mathsf{H}}\subseteq\mbox{ker }{\mathsf{S}}^{\top}$;*

(b)

${\mathsf{R}}-{\mathsf{S}}^{\top}{\mathsf{H}}^{\dagger}{\mathsf{S}}$* is *$({\mathbb{R}}^{p}_{+}-)$copositive.

Here ${\mathsf{H}}^{\dagger}$ is the Moore-Penrose pseudoinverse of ${\mathsf{H}}$.

**Proof. **The argument is an easy extension of the arguments that led to [9, Thm.3.1]. $\Box$

3 Relaxations for extended trust region problems

The problem we study here is given by

[TABLE]

Throughout this paper, we assume that the feasible set of problem (P) is non-empty. The model problem (P) can be reformulated as

[TABLE]

where $F:=\left\{{\mathsf{x}}\in{\mathbb{R}}^{n}:f_{i}({\mathsf{x}})\leq 0\,,\,i=1,2\right\}$, ${\mathsf{b}}\in{\mathbb{R}}^{p}$ and ${\mathsf{B}}$ is a $p\times n$ matrix.

For our approach, it will be convenient to introduce slack variables $s_{j}:=b_{j}-({\mathsf{B}}{\mathsf{x}})_{j}$ for all $j{\in\![{1}\!:\!{p}]}$, arriving at new primal-feasible points ${\mathsf{y}}=({\mathsf{s}},{\mathsf{x}})\in{\mathbb{R}}^{p}_{+}\times{\mathbb{R}}^{n}$, in other words, to replace $P$ with

[TABLE]

with the $p\times(p+n)$-matrix $\bar{\mathsf{B}}:=[{\mathsf{I}}_{p}\,|\,{\mathsf{B}}]$.

We now need to extend all original functions in the obvious way, namely $\bar{f}_{i}({\mathsf{y}})=\bar{f}_{i}({\mathsf{s}},{\mathsf{x}})=f_{i}({\mathsf{x}})$ by writing $\bar{\mathsf{Q}}_{i}={\mathsf{O}}\oplus{\mathsf{Q}}_{i}$, i.e., adding $p$ zero rows and $p$ zero columns to ${\mathsf{Q}}_{i}$, arriving at symmetric matrices of order $p+n$; likewise we define $\bar{\mathsf{q}}_{i}^{\top}=[{\mathsf{o}}^{\top},{\mathsf{q}}_{i}^{\top}]$. Finally, by introducing another quadratic constraint, defining $\bar{\mathsf{Q}}_{3}=\bar{\mathsf{B}}^{\top}\bar{\mathsf{B}}$, $\bar{\mathsf{q}}_{3}=\bar{\mathsf{B}}^{\top}{\mathsf{b}}$ and $c_{3}={\mathsf{b}}^{\top}{\mathsf{b}}$, we rephrase the $p$ linear constraints $\bar{\mathsf{B}}{\mathsf{y}}={\mathsf{b}}$ into one quadratic constraint $\bar{f}_{3}({\mathsf{y}})={\|\bar{\mathsf{B}}{\mathsf{y}}-{\mathsf{b}}\|}^{2}{\leq 0}$.

In this way, the original problem (3) is rephrased in a somehow standardized form, namely

[TABLE]

The optimal value $z^{*}$ of (3) need not be attained, and it could also be equal to $-\infty$ (in the unbounded case) or to $+\infty$ (in the infeasible case). Considering ${\mathsf{Q}}_{i}={\mathsf{O}}$ would also allow for linear inequality constraints. But it is often advisable to discriminate the functional form of constraints, and we will adhere to this principle in what follows.

Copositive relaxation

Next, we introduce a copositive relaxation for (P). Let ${\mathsf{y}}=({\mathsf{s}},{\mathsf{x}})\in\mathbb{R}^{p}\times\mathbb{R}^{n}$. Now consider multipliers ${\mathsf{u}}\in{\mathbb{R}}^{3}_{+}$ of the inequality constraints $\bar{f}_{i}({\mathsf{y}})=f_{i}({\mathsf{x}})\leq 0$, $i{\in\![{1}\!:\!{3}]}$, and ${\mathsf{v}}\in{\mathbb{R}}^{p}_{+}$ for the sign constraints ${\mathsf{s}}\in{\mathbb{R}}^{p}_{+}$. Then we define the full Lagrangian function for problem (4) as

[TABLE]

Let $\Upsilon={\mathbb{R}}^{p+1}_{+}\times{\mathbb{R}}^{n}$. Recall that the matrix ${\mathsf{J}}_{0}$ and the Shor relaxation matrix ${\mathsf{M}}(q)$ for a quadratic function $q$ are given as in (1) and (2) respectively. Then the matrix ${\mathsf{M}}(L(\cdot;{\mathsf{u}},{\mathsf{o}}))-\mu{\mathsf{J}}_{0}$ can be written as below:

[TABLE]

where ${\mathsf{H}}_{u}={\mathsf{Q}}_{0}+u_{1}{\mathsf{Q}}_{1}+u_{2}{\mathsf{A}}^{\top}{\mathsf{A}}$ is the Hessian of the Lagrangian function. We now associate a copositive relaxation for (P) as follows:

[TABLE]

It is worth noting that, unlike in [7], our model problem does not require the variables to be nonnegative, and so the techniques in constructing a copositive relaxation as in [7] cannot be applied directly. Here we achieve this task by introducing nonnegative slack variables.

An important observation is that the copositive relaxation can be equivalently reformulated as a semi-Lagrangian dual problem of the problem (P). Recall that the usual Lagrangian dual (or Lagrangian relaxation) of (P) is given by

[TABLE]

where $\Theta({\mathsf{u}},{\mathsf{v}}):=\inf\left\{L({\mathsf{y}};{\mathsf{u}},{\mathsf{v}}):{\mathsf{y}}\in{\mathbb{R}}^{p+n}\right\}$. A form of partial Lagrangian relaxation called semi-Lagrangian of (P) (see [7] and the references therein) is given by

[TABLE]

where $\Theta_{\rm semi}({\mathsf{u}}):=\inf\left\{L({\mathsf{y}};{\mathsf{u}},{\mathsf{o}}):{\mathsf{y}}\in{\mathbb{R}}^{p}_{+}\times{\mathbb{R}}^{n}\right\}$. The relation between copositive relaxation, full and semi-Lagrangian bounds can be summarized in the following chain of inequalities

[TABLE]

We note that the relation $z_{\rm LD}^{*}\leq z_{\rm semi}^{*}\leq z^{*}$ follows by the construction, and the equality $z_{\rm COP}^{*}=z_{\rm semi}^{*}$ follows by adapting the techniques in [7, Lemma 2.1] to the polyhedral cone ${\mathbb{R}}^{p}_{+}\times{\mathbb{R}}^{n}$ (see also (11) later for a detailed proof).

We now illustrate that, in general, a copositive relaxation can provide a much tighter bound for the model problem (P) than the usual Lagrangian dual. Indeed, in the following example, we see that the copositive relaxation is tight while the usual Lagrangian dual yields a trivial lower bound which has infinite gap. As we will see later (Proposition 4.3), one can indeed construct a whole class of quadratic optimization problem with exact copositive relaxation but infinite Lagrangian duality gap.

Example 3.1

(Copositive relaxation vs Lagrangian relaxation)* Consider the following nonconvex quadratic optimization problem with simple linear inequality constraints*

[TABLE]

Clearly, the objective function is not convex and the optimal value of this problem is $z^{*}=0$. We next observe that this problem can be converted to our standard form as

[TABLE]

Then the copositive relaxation reads

[TABLE]

Clearly, from the copositivity requirement, $z_{\rm COP}^{*}\leq 0$. Moreover, it can be verified from Lemma 2.1 that, for $\mu=0$ and $u=1$, the matrix

[TABLE]

Thus, $z_{\rm COP}^{*}=z^{*}=0$.

Next we show that $z_{\rm LD}^{*}=-\infty$. To see this, we only need to show that for each fixed $u\geq 0$ and ${\mathsf{v}}=(v_{1},v_{2})^{\top}\in{\mathbb{R}}_{+}^{2}$, we have

[TABLE]

Indeed, taking ${\mathsf{x}}=(-t,t)$ and ${\mathsf{s}}=(t,-t)$ we see that, as $t\rightarrow+\infty$ ,

[TABLE]

4 Tightness of copositive relaxation

We consider, for ${\mathsf{y}}=({\mathsf{s}},{\mathsf{x}})\in{\mathbb{R}}^{p}_{+}\times{\mathbb{R}}^{n}$, the full Lagrangian function

[TABLE]

As in [7], let us say that the pair $({\mathsf{x}};{\mathsf{u}},{\mathsf{v}})\in(F\cap P)\times{\mathbb{R}}^{3}_{+}\times{\mathbb{R}}^{p}$ is a generalized KKT pair for (3) if and only if, for ${\mathsf{s}}={\mathsf{b}}-{\mathsf{B}}{\mathsf{x}}$ and ${\mathsf{y}}=({\mathsf{s}},{\mathsf{x}})$, it satisfies both the first-order conditions $\nabla_{\mathsf{y}}L({\mathsf{y}};{\mathsf{u}},{\mathsf{v}})={\mathsf{o}}$ and as well the complementarity conditions $v_{k}s_{k}=0$ for all $k{\in\![{1}\!:\!{p}]}$ and $u_{i}\bar{f}_{i}({\mathsf{y}})=0$ for all $i{\in\![{1}\!:\!{3}]}$, but without requiring $v_{k}\geq 0$.

Geometric conditions for exact copositive relaxation

Next, we provide a geometric condition ensuring the exactness of the copositive relaxation which does not rely on the information of KKT pairs. To do this, denote ${\mathsf{y}}=({\mathsf{s}},{\mathsf{x}})$ and let $\bar{f}_{0}({\mathsf{y}})={\mathsf{x}}^{\top}{\mathsf{Q}}_{0}{\mathsf{x}}+2{\mathsf{q}}_{0}^{\top}{\mathsf{x}}$, $\bar{f}_{1}({\mathsf{y}})={\mathsf{x}}^{\top}{\mathsf{Q}}_{1}x+2{\mathsf{q}}_{1}^{\top}{\mathsf{x}}-1$, $\bar{f}_{2}({\mathsf{y}})={\|{\mathsf{A}}{\mathsf{x}}-{\mathsf{a}}\|}^{2}-1$ and $\bar{f}_{3}({\mathsf{y}})=\|{\mathsf{B}}{\mathsf{x}}+{\mathsf{s}}-{\mathsf{b}}\|^{2}$.

Theorem 4.1

For the extended trust region problem (P), let

[TABLE]

Suppose that $\Omega$ is closed and convex. Then we have $z_{\rm COP}^{*}=z^{*}$.

**Proof. ** Let $z_{\rm semi}^{*}$ denote the optimal value of the semi-Lagrangian dual (8). We first observe that $z_{\rm COP}^{*}=z_{\rm semi}^{*}$. To see this, for any $\mu\in{\mathbb{R}}$ and any quadratic function $q$ defined on $\mathbb{R}^{p+n}$, it can be directly verified that the following two conditions are equivalent:

(a)

$q({\mathsf{y}})\geq\mu$ for all ${\mathsf{y}}\in{\mathbb{R}}^{p}_{+}\times{\mathbb{R}}^{n}$;

(b)

the $(n+p+1)\times(n+p+1)$-matrix ${\mathsf{M}}(q-\mu)={\mathsf{M}}(q)-\mu{\mathsf{J}}_{0}$ is $\Upsilon$-copositive.

This equivalence implies the identity

[TABLE]

Note that above equality holds, by the usual convention that $\sup\emptyset=-\infty$, also if $q$ is unbounded from below on ${\mathbb{R}}^{p}_{+}\times{\mathbb{R}}^{n}$. Applying (10) with $q=L(\cdot;{\mathsf{u}},{\mathsf{o}})$, we see that

[TABLE]

Then it follows from the definitions of semi-Lagrangian dual and copositive relaxation that

[TABLE]

As $z^{*}\geq z_{\rm semi}^{*}$, we see that $z^{*}\geq z_{\rm COP}^{*}$ always holds. So, we can assume without loss of generality that $z^{*}>-\infty$. As the feasible set of (P) is nonempty, we have $z^{*}<+\infty$, and hence $z^{*}\in\mathbb{R}$. Let $\epsilon>0$. Thus $[z^{*}-\epsilon,0,0,0]^{\top}\notin\Omega$. By the strict separation theorem, there exists $(\mu_{0},\mu_{1},\mu_{2},\mu_{3})\neq(0,0,0,0)$ such that

[TABLE]

As $\Omega+\mathbb{R}^{4}_{+}\subseteq\Omega$, we get $\mu_{i}\geq 0$ for all $i{\in\![{0}\!:\!{3}]}$. Moreover, by the feasibility, we see that $\mu_{0}>0$. Thus, by dividing $\mu_{0}$ on both sides, we see that for all ${\mathsf{y}}=({\mathsf{s}},{\mathsf{x}})\in\mathbb{R}^{p}_{+}\times\mathbb{R}^{n}$

[TABLE]

where $\lambda_{i}=\mu_{i}/\mu_{0}$, $i=1,2,3$. This implies that

[TABLE]

where the second inequality follows from the definition of semi-Lagrangian dual (8). By letting $\epsilon\searrow 0$, we have $z^{*}\leq z_{\rm COP}^{*}$. As the reverse inequality always holds, the conclusion follows. $\Box$

Before we provide simple sufficient conditions ensuring this geometrical condition, we will illustrate it using our previous example.

Example 4.1

Consider the same example as in Example 3.1. We observe that, in this case ${\mathsf{A}},{\mathsf{Q}}_{1}$ are zero matrices and ${\mathsf{q}}_{1},{\mathsf{a}}$ are zero vectors, and so, the set $\Omega$ becomes

[TABLE]

Then

[TABLE]

where

[TABLE]

Now we provide an analytic expression for $\Omega_{1}$. Note that, if $x_{1}=0$, then $[\frac{x_{1}^{2}}{2}+2x_{1}x_{2}+x_{2}^{2},(x_{1}-s_{1})^{2}+(x_{2}-s_{2})^{2}]^{\top}\in\mathbb{R}_{+}^{2}$ and $\mathbb{R}^{2}_{+}\subseteq\Omega_{1}$ (take $x_{2}=s_{2}\geq 0$ and $s_{1}\geq 0=x_{1}$ to get an arbitrary point $(x_{2}^{2},s_{1}^{2})^{\top}\in{\mathbb{R}}^{2}_{+}$). Thus we only need to consider the case where $x_{1}\neq 0$. Then

[TABLE]

where the last equality follows by noting that $\min_{s\geq 0}(x-s)^{2}=\min\{x,0\}^{2}$. Direct verification now shows that

[TABLE]

which is closed and convex. Therefore, $\Omega$ is also closed and convex.

Next, we provide some verifiable sufficient conditions guaranteeing convexity as well as closedness of $\Omega$. To do this, recall that an $n\times n$ matrix ${\mathsf{M}}$ is called a $Z$-matrix if its off-diagonal elements $M_{ij}$ with $1\leq i,j\leq n$ and $i\neq j$, are all non-positive. We also need the following joint-range convexity for $Z$-matrices.

Lemma 4.1

Let ${\mathsf{M}}_{i}$, $i{\in\![{1}\!:\!{q}]}$, be symmetric $Z$-matrices of order $n$. Then

[TABLE]

is a convex cone.

**Proof. ** The proof is similar to [19, Theorem 5.1]. $\Box$

Proposition 4.1

Suppose that ${\mathsf{Q}}_{0},{\mathsf{Q}}_{1}$, ${\mathsf{A}}^{\top}{\mathsf{A}}$ are all $Z$-matrices, ${\mathsf{B}}=-{\mathsf{I}}_{n}$ and ${\mathsf{q}}_{0},{\mathsf{q}}_{1},{\mathsf{a}},{\mathsf{b}}$ are zero vectors. Then $\Omega$ is convex.

**Proof. ** Let $\bar{h}_{0}({\mathsf{y}})={\mathsf{x}}^{\top}{\mathsf{Q}}_{0}{\mathsf{x}}$, $\bar{h}_{1}({\mathsf{y}})={\mathsf{x}}^{\top}{\mathsf{Q}}_{1}{\mathsf{x}}$, $\bar{h}_{2}({\mathsf{y}})={\|{\mathsf{A}}{\mathsf{x}}\|}^{2}$ and $\bar{h}_{3}({\mathsf{y}})=\|{\mathsf{x}}-{\mathsf{s}}\|^{2}$ with ${\mathsf{y}}=({\mathsf{s}},{\mathsf{x}})$, so $p=n$ here. We first note that $\Omega=(0,-1,-1,0)+\bar{\Omega}$ where

[TABLE]

To see the convexity of $\Omega$, it suffices to show that $\bar{\Omega}$ is convex. To verify this, take $(u_{0},u_{1},u_{2},u_{3})\in\bar{\Omega}$ and $(v_{0},v_{1},v_{2},v_{3})\in\bar{\Omega}$, and let $\lambda\in[0,1]$. Then there exist $(\hat{{\mathsf{s}}},\hat{{\mathsf{x}}})\in\mathbb{R}^{n}_{+}\times\mathbb{R}^{n}$ and $(\tilde{{\mathsf{s}}},\tilde{{\mathsf{x}}})\in\mathbb{R}^{n}_{+}\times\mathbb{R}^{n}$ such that

[TABLE]

In particular, $u_{3}\geq 0$ and $v_{3}\geq 0$. We now verify that

[TABLE]

Note that $\bar{h}_{i}({\mathsf{y}})={\mathsf{y}}^{\top}\left[{\mathsf{O}}\oplus{\mathsf{Q}}_{i}\right]{\mathsf{y}}$ for $i\in\left\{0,1\right\}$ and $\bar{h}_{2}({\mathsf{y}})={\mathsf{y}}^{\top}\left[{\mathsf{O}}\oplus{\mathsf{A}}^{\top}{\mathsf{A}}\right]{\mathsf{y}}$, cf. (1), while

[TABLE]

so that the associated matrices

[TABLE]

are all $Z$-matrices. We see that

[TABLE]

is convex. So there exists $({\mathsf{r}},{\mathsf{z}})\in\mathbb{R}^{p}\times\mathbb{R}^{n}$ such that

[TABLE]

Denote ${\mathsf{z}}=(z_{1},\ldots,z_{n})$ and let $|{\mathsf{z}}|=(|z_{1}|,\ldots,|z_{n}|)$. The $Z$-matrices assumptions ensure

[TABLE]

[TABLE]

and

[TABLE]

Therefore, $\lambda(u_{0},u_{1},u_{2},u_{3})+(1-\lambda)(v_{0},v_{1},v_{2},v_{3})\in\bar{\Omega}$, and so the conclusion follows. $\Box$

Proposition 4.2

Suppose that there exist $\tau_{i}\geq 0$, $i{\in\![{0}\!:\!{2}]}$, such that

[TABLE]

Then $\Omega$ is closed.

**Proof. ** Let ${\mathsf{r}}^{(k)}\in\Omega$ such that ${\mathsf{r}}^{(k)}\rightarrow{\mathsf{r}}\in{\mathbb{R}}^{4}$. Then there exists ${\mathsf{y}}_{k}=({\mathsf{s}}_{k},{\mathsf{x}}_{k})\in{\mathbb{R}}^{p}_{+}\times{\mathbb{R}}^{n}$ such that

[TABLE]

We first see that $\{{\mathsf{x}}_{k}\}$ is bounded. To see this, note that

[TABLE]

Since \nabla^{2}\big{(}\sum_{i=0}^{2}\tau_{i}f_{i}\big{)}({\mathsf{x}})\equiv\tau_{0}{\mathsf{Q}}_{0}+\tau_{1}{\mathsf{Q}}_{1}+\tau_{2}{\mathsf{A}}^{\top}{\mathsf{A}}\succ 0, this implies that $\{{\mathsf{x}}_{k}\}$ must be bounded. Taking into account that

[TABLE]

it follows that also $\{{\mathsf{s}}_{k}\}$ is a bounded sequence. By passing to subsequences, we may assume that ${\mathsf{y}}_{k}=({\mathsf{s}}_{k},{\mathsf{x}}_{k})\rightarrow({\mathsf{s}},{\mathsf{x}})=:{\mathsf{y}}\in{\mathbb{R}}^{p}_{+}\times{\mathbb{R}}^{n}$. Passing to the limit, we see that $\bar{f}_{i}({\mathsf{y}})\leq r_{i},\ i{\in\![{0}\!:\!{3}]}$ and so ${\mathsf{r}}\in\Omega$. Thus $\Omega$ is closed. $\Box$

Sufficient global optimality conditions

Now, we obtain the following sufficient second-order global optimality condition, which also implies that the copositive relaxation is tight, generalizing a recent result [9, Section 6.3] for CDT problems:

Theorem 4.2

If at a generalized KKT pair $(\bar{\mathsf{x}};\bar{\mathsf{u}},\bar{\mathsf{v}})\in(F\cap P)\times{\mathbb{R}}^{3}_{+}\times{\mathbb{R}}^{p}$ of problem (3), we have

[TABLE]

then $\bar{\mathsf{x}}$ is a globally optimal solution to (3) and $z^{*}=z_{\rm COP}^{*}$.

**Proof. **We first note that the conic dual of problem (6) is

[TABLE]

with ${\mathsf{M}}_{i}={\mathsf{M}}(\bar{f}_{i})$ and ${\mathcal{C}}_{\Upsilon}={\mbox{conv }}\left\{{\mathsf{x}}{\mathsf{x}}^{\top}:{\mathsf{x}}\in{\mathbb{R}}^{p+1}_{+}\times{\mathbb{R}}^{n}\right\}$. From standard conical lifting and weak duality arguments it follows

[TABLE]

Let $\bar{\mathsf{s}}={\mathsf{b}}-{\mathsf{B}}\bar{\mathsf{x}}$ and $\bar{\mathsf{y}}=(\bar{\mathsf{s}},\bar{\mathsf{x}})$. The complementarity conditions imply $\bar{\mathsf{v}}^{\top}\bar{\mathsf{s}}=0$ and $\sum\limits_{i=1}^{3}u_{i}\bar{f}_{i}(\bar{\mathsf{y}})=0$, so that both the standard $L(\bar{\mathsf{y}};\bar{\mathsf{u}},\bar{\mathsf{v}})=f_{0}(\bar{\mathsf{x}})$ and as well $L(\bar{\mathsf{y}};\bar{\mathsf{u}},{\mathsf{o}})=f_{0}(\bar{\mathsf{x}})$, which will be used now. Indeed, put $\bar{\mathsf{z}}^{\top}=[1,\bar{\mathsf{y}}^{\top}]$ and $\bar{\mathsf{X}}=\bar{\mathsf{z}}\bar{\mathsf{z}}^{\top}\in{\mathcal{C}}_{\Upsilon}$. Then from the definition of $\bar{\mathsf{S}}$ we get

[TABLE]

so that $(\bar{\mathsf{X}},\bar{\mathsf{S}})$ form an optimal primal-dual pair for the copositive problem (15) and (6) with zero duality gap. We conclude, by feasibility of $\bar{\mathsf{x}}$ and definition of $z^{*}$, and because of (6) with $\mu=f_{0}(\bar{\mathsf{x}})$, cf. (14),

[TABLE]

yielding tightness of the copositive relaxation, zero duality gap for the copositive-cp conic optimization problems, and optimality of $\bar{\mathsf{x}}$. $\Box$

While checking copositivity is NP-hard, the slack matrix $\bar{\mathsf{S}}$ may lie in a slightly smaller but tractable approximation cone, and then global optimality is guaranteed even in cases where $\bar{\mathsf{S}}$ is indefinite. The difference can also be expressed in properties of the Hessian ${\mathsf{H}}_{\bar{\mathsf{u}}}$ of the Lagrangian (recall that this is the same irrespective of our decision whether to relax also the linear constraints or not): indeed, a similar condition on the slack matrix yielding tightness of the classical Lagrangian bound (i.e. $z_{\rm LD}^{*}=z^{*}$) or the equivalent SDP relaxation [7, Section 5.1] implies that its lower right principal submatrix ${\mathsf{H}}_{\bar{\mathsf{u}}}$ has to be psd, and we know this is too strong in some cases [39].

By contrast, the condition $\bar{\mathsf{S}}\in{\mathcal{C}}^{\star}_{\Upsilon}$ (giving tightness $z_{\rm COP}^{*}=z^{*}$), by the same argument using Lemma 2.1 and (5), only yields positive semidefiniteness of ${\mathsf{H}}_{\bar{\mathsf{u}}}+\bar{\mathsf{u}}_{3}{\mathsf{B}}{\mathsf{B}}^{\top}$. Of course, this happens with higher frequency than positive-definiteness of the Hessian, and the discrepancy is not negligible, see [7, Section 5] for an example.

We note that Theorem 4.2 can be used to construct a class of problems where copositive relaxation is always tight while the usual Lagrangian dual produces a trivial bound with infinite duality gaps. To see this, we shall need the following auxiliary result.

Lemma 4.2

Let ${\mathsf{M}}$ be strictly ${\mathbb{R}}^{n}_{+}$-copositive; then there exists a constant $\sigma>0$ such that

[TABLE]

**Proof. **We first note that the conclusion trivially holds if ${\mathsf{M}}$ is further assumed to be positive semidefinite. So we may assume without loss of generality that $\lambda_{\rm min}({\mathsf{M}})<0$. For any ${\mathsf{x}}\in{\mathbb{R}}^{n}$, denote by

[TABLE]

and by ${\mathsf{x}}^{-}:={\mathsf{x}}^{+}-{\mathsf{x}}\in{\mathbb{R}}^{n}_{+}$ so that ${\mathsf{x}}={\mathsf{x}}^{+}-{\mathsf{x}}^{-}$. Furthermore, we have $\|{\mathsf{x}}-{\mathsf{s}}\|\geq\|{\mathsf{x}}^{-}\|$ for all ${\mathsf{s}}\in{\mathbb{R}}^{n}_{+}$, as can be seen easily. Therefore we are done if we establish the (non-quadratic) inequality ${\mathsf{x}}^{\top}{\mathsf{M}}{\mathsf{x}}+\sigma\|{\mathsf{x}}^{-}\|^{2}>0$ whenever ${\mathsf{x}}\in{\mathbb{R}}^{n}\setminus\left\{{\mathsf{o}}\right\}$. Now, given ${\mathsf{M}}$ is strictly copositive, we choose $\rho:=\min\left\{{\mathsf{x}}^{\top}{\mathsf{M}}{\mathsf{x}}:{\mathsf{x}}\in{\mathbb{R}}^{n}_{+},\|{\mathsf{x}}\|=1\right\}>0$. Note that ${\mathsf{x}}\in{\mathbb{R}}^{n}_{+}$ if and only if $\|{\mathsf{x}}^{-}\|=0$. By continuity and compactness, we infer existence of an $\varepsilon>0$ such that ${\mathsf{x}}^{\top}{\mathsf{M}}{\mathsf{x}}\geq\frac{\rho}{2}$ whenever $\|{\mathsf{x}}^{-}\|\leq\varepsilon$ and $\|{\mathsf{x}}\|=1$. Now we distinguish two cases:

Case 1: $\|{\mathsf{x}}^{-}\|\geq\varepsilon\|{\mathsf{x}}\|>0$. In this case, we have

[TABLE]

where we set $\sigma:=-2\lambda_{\rm min}({\mathsf{M}})/\varepsilon^{2}>0$;

Case 2: $\|{\mathsf{x}}^{-}\|\leq\varepsilon\|{\mathsf{x}}\|$. In this case, one has ${\mathsf{x}}^{\top}{\mathsf{M}}{\mathsf{x}}\geq\frac{\rho}{2}{\|{\mathsf{x}}\|}^{2}>-\sigma\|{\mathsf{x}}^{-}\|^{2}$ (for any $\sigma>0$).

So in both cases, we obtain ${\mathsf{x}}^{\top}{\mathsf{M}}{\mathsf{x}}+\sigma{\|{\mathsf{x}}^{-}\|}^{2}>0$ for all ${\mathsf{x}}\in{\mathbb{R}}^{n}\setminus\left\{{\mathsf{o}}\right\}$, and the lemma is shown. $\Box$

Consider the following non-convex quadratic optimization problem

[TABLE]

where ${\mathsf{Q}}_{0}$ is a strictly $\mathbb{R}^{n}_{+}$-copositive and indefinite matrix. This problem can be regarded as a special case of the model problem (P) with ${\mathsf{Q}}_{1}={\mathsf{I}}_{n}$, ${\mathsf{A}}={\mathsf{O}}$ and ${\mathsf{a}}=0$. We now see that this class of quadratic optimization admits a tight copositive relaxation and an infinite Lagrangian duality gap.

Proposition 4.3

(Tight copositive relaxation and infinite Lagrangian duality gap for (EP))* For problem (EP), let $z^{*}$, $z_{\rm LD}^{*}$ and $z_{\rm COP}^{*}$ denote the optimal value of (EP), the Lagrangian relaxation of (EP) and copositive relaxation of (EP) respectively. Then $z_{\rm LD}^{*}=-\infty$ and $z^{*}=z_{\rm COP}^{*}=0$.*

**Proof. ** Direct verification shows that ${\mathsf{o}}\in\mathbb{R}^{n}$ is a global solution with the optimal value $z^{*}=0$. We first observe that, as ${\mathsf{Q}}_{0}$ is indefinite, the optimal value of the Lagrangian dual is $z_{\rm LD}^{*}=-\infty$. Next, as ${\mathsf{Q}}_{0}$ is strictly $\mathbb{R}^{n}_{+}$-copositive, the preceding lemma implies that there exists $\sigma>0$ such that

[TABLE]

Let $\bar{\mathsf{y}}=(\bar{\mathsf{x}},\bar{\mathsf{s}})$ with $\bar{\mathsf{x}}=\bar{\mathsf{s}}={\mathsf{o}}\in\mathbb{R}^{n}$, $\bar{\mathsf{u}}=(0,0,\sigma)\in\mathbb{R}^{3}_{+}$ and $\bar{\mathsf{v}}={\mathsf{o}}\in\mathbb{R}^{n}$. Then we see that $(\bar{\mathsf{y}};\bar{\mathsf{u}},\bar{\mathsf{v}})$ is a (generalized) KKT pair for (EP). Moreover, we have

[TABLE]

For all ${\mathsf{d}}:=(r,{\mathsf{s}},{\mathsf{x}})\in\Upsilon=\mathbb{R}_{+}\times{\mathbb{R}}^{n}_{+}\times{\mathbb{R}}^{n}$, above implies

[TABLE]

and hence ${\mathsf{M}}(L(\cdot;\bar{\mathsf{u}},{\mathsf{o}}))-f_{0}(\bar{\mathsf{x}}){\mathsf{J}}_{0}$ is $\Upsilon$-copositive. This shows that $z_{\rm COP}^{*}=0$. $\Box$

5 Relaxation tightness in extended CDT Problems

In this section, we examine the so-called extended CDT problem:

[TABLE]

This problem is a special case of our general model problem with ${\mathsf{Q}}_{1}={\mathsf{I}}_{n}$ and ${\mathsf{q}}_{1}={\mathsf{o}}$. In the cases where no linear inequalities are present, the problem $\mathrm{(P_{CDT})}$ reduces to the so-called CDT problem (also referred as two-ball trust region problems, TTR). The CDT problem, in general, is much more challenging than the well-studied trust region problems and has received much attention lately, see for example [1, 5, 6, 9, 14]. The problem $\mathrm{(P_{CDT})}$ arises from robust optimization [20] as well as applying trust region techniques for solving nonlinear optimization problems with both nonlinear and linear constraints: see [34] for the case of trust region problems with additional linear inequalities and see [6, 9] for the case of CDT problems. We will establish simple conditions ensuring exactness of the copositive relaxations and the usual Lagrangian relaxations of the extended CDT problem.

First of all, we note that the sufficient second-order global optimality condition in Theorem 4.2, specialized to the setting $\mathrm{(P_{CDT})}$, yields the exact copositive relaxation for extended CDT problems.

Corollary 5.1

Let $(\bar{\mathsf{x}};\bar{\mathsf{u}},\bar{\mathsf{v}})\in F_{\rm CDT}\times{\mathbb{R}}^{3}_{+}\times{\mathbb{R}}^{p}$ be a generalized KKT pair of problem $\mathrm{(P_{CDT})}$ where $F_{\rm CDT}$ is the feasible set of $\mathrm{(P_{CDT})}$. Denote by $\bar{\mu}:=\bar{\mathsf{x}}^{\top}{\mathsf{Q}}_{0}\bar{\mathsf{x}}+2{\mathsf{q}}_{0}^{\top}\bar{\mathsf{x}}$. Suppose that

[TABLE]

is $(\mathbb{R}^{p+1}_{+}\times\mathbb{R}^{n})$-copositive. Then $\bar{\mathsf{x}}$ is a globally optimal solution to problem $\mathrm{(P_{CDT})}$ and $z^{*}=z_{\rm COP}^{*}$.

**Proof. ** The conclusion follows by Theorem 4.2 with ${\mathsf{Q}}_{1}={\mathsf{I}}_{n}$ and ${\mathsf{q}}_{1}={\mathsf{o}}$. $\Box$

Next we examine when the usual Lagrangian relaxation is exact for the extended CDT problems. To this end, we define an auxiliary convex optimization problem

[TABLE]

where

[TABLE]

We first see that if the auxiliary convex problem (AP) has a minimizer on the sphere $\{{\mathsf{x}}\in{\mathbb{R}}^{n}:\|{\mathsf{x}}\|=1\}$, then an extended CDT problem has a tight semi-Lagrangian relaxation. We will provide a sufficient condition in terms of the original data guaranteeing this condition later (in Theorem 4.1).

Lemma 5.1

Suppose that the auxiliary convex problem (AP) has a minimizer on the sphere $\{{\mathsf{x}}:\|{\mathsf{x}}\|=1\}$. Then $z_{\rm LD}^{*}=z_{\rm COP}^{*}=z^{*}$.

**Proof. ** Recall that $z_{\rm LD}^{*}\leq z_{\rm COP}^{*}\leq z^{*}$. So it suffices to show that $z_{\rm LD}^{*}=z^{*}$. Without loss of generality, we assume that $\lambda_{\min}({\mathsf{Q}}_{0})<0$ (otherwise $\mathrm{(P_{CDT})}$ is a convex quadratic problem and so $z_{\rm LD}^{*}=z^{*}$). Let ${\mathsf{x}}^{*}$ be a solution of (AP) with $\|{\mathsf{x}}^{*}\|=1$. As $\lambda_{\min}({\mathsf{Q}}_{0})<0$, it follows from $\|{\mathsf{x}}\|\leq 1$ for all ${\mathsf{x}}\in F_{\rm CDT}$ that

[TABLE]

where the last inequality follows from feasibility of ${\mathsf{x}}^{*}$ for the extended CDT problem. This shows that $z^{*}={\rm val}\mathrm{(AP)}+\lambda_{\min}({\mathsf{Q}}_{0})$. Rewriting (AP) as

[TABLE]

we obtain the Lagrangian dual of this problem which can be stated as

[TABLE]

where $\tilde{\mathsf{M}}(\mu,{\mathsf{u}},{\mathsf{v}})$ denotes the matrix

[TABLE]

Note that the feasible set of (AP1) is bounded by $\|{\mathsf{x}}\|\leq 1$ and ${\mathsf{s}}={\mathsf{B}}{\mathsf{x}}-{\mathsf{b}}$ for all feasible $({\mathsf{x}},{\mathsf{s}})$. Since any convex optimization problem with compact feasible set enjoys a zero duality gap (for example see [21]), it follows that

[TABLE]

Finally, the conclusion results by noting that ${\rm val}\mathrm{(LD1)}=z_{\rm LD}^{*}+\lambda_{\min}({\mathsf{Q}}_{0})$. So we have $z^{*}=z_{\rm LD}^{*}$ and furthermore $z_{\rm LD}^{*}=z_{\rm COP}^{*}=z^{*}$. $\Box$

Next we provide a simple sufficient condition formulated in terms of the original data guaranteeing tightness of the relaxations. It is important to note that this sufficient condition can be efficiently verified by solving a feasibility problem of a linear optimization problem.

Theorem 5.1

Let ${\mathsf{M}}=[{\mathsf{Q}}_{0}^{+}|{\mathsf{A}}^{\top}]^{\top}$. Suppose that

[TABLE]

Then $z_{\rm LD}^{*}=z_{\rm COP}^{*}=z^{*}$.

**Proof. ** By the preceding lemma, the conclusion follows if we show that the auxiliary convex problem (AP) has a minimizer on the sphere $\{{\mathsf{x}}:\|{\mathsf{x}}\|=1\}$. Suppose that a minimizer ${\mathsf{x}}^{*}$ of (AP) satisfies $\|{\mathsf{x}}^{*}\|<1$. Let ${\mathsf{v}}\in{\rm ker}({\mathsf{M}})\cap\{{\mathsf{v}}\in\mathbb{R}^{n}:{\mathsf{B}}{\mathsf{v}}\leq{\mathsf{o}}\}\cap\{{\mathsf{v}}:{\mathsf{q}}_{0}^{\top}{\mathsf{v}}\geq 0\}$ with ${\mathsf{v}}\neq{\mathsf{o}}$. Consider ${\mathsf{x}}(t)={\mathsf{x}}^{*}+t{\mathsf{v}}$, $t\geq 0$. Then there exists $t_{0}>0$ such that $\|{\mathsf{x}}(t_{0})\|=1$. Now observe

[TABLE]

[TABLE]

and

[TABLE]

This shows that ${\mathsf{x}}(t_{0})$ is a minimizer for (AP) and $\|{\mathsf{x}}(t_{0})\|=1$. The conclusion follows. $\Box$

Remark 5.1** (LP reformulation of the sufficient condition (17))**

Our sufficient condition (17) can be efficiently verified by determining a feasible solution to the following LP

[TABLE]

Remark 5.2

(Links to the known dimension condition for exact relaxation)* In the special case where ${\mathsf{A}}={\mathsf{O}}$ and ${\mathsf{a}}={\mathsf{o}}\in\mathbb{R}^{n}$, the authors showed in [20], that under the dimension condition*

[TABLE]

where ${\mathsf{b}}_{i}^{\top}$ is the $i$th row of ${\mathsf{B}}$, the SDP (or Lagrangian) relaxation is exact. We observe that this dimension condition is strictly stronger than our sufficient condition in the preceding theorem.

Firstly, we see that the dimension condition implies our sufficient condition in the preceding theorem. To see this, suppose the above dimension condition holds. Then there exists ${\mathsf{v}}\neq{\mathsf{o}}$ such that ${\mathsf{v}}\in{\rm ker}{\mathsf{Q}}_{0}^{+}$ and ${\mathsf{b}}_{i}^{\top}{\mathsf{v}}=0$ for all $i{\in\![{1}\!:\!{p}]}$ (and hence ${\mathsf{B}}{\mathsf{v}}={\mathsf{o}}$). By replacing ${\mathsf{v}}$ by $-{\mathsf{v}}$ if necessary, we can assume that ${\mathsf{q}}_{0}^{\top}{\mathsf{v}}\geq 0$. Thus, our sufficient condition in the preceding theorem holds.

To see the dimension condition is strictly stronger, let us consider {\mathsf{Q}}_{0}=\left[\begin{array}[]{cc}2&0\\ 0&-2\end{array}\right], {\mathsf{A}}=\left[\begin{array}[]{cc}0&0\\ 0&0\end{array}\right], {\mathsf{q}}_{0}={\mathsf{a}}=\left[\begin{array}[]{c}0\\ 0\end{array}\right] and ${\mathsf{B}}={\mathsf{b}}_{1}^{\top}=[-1,0]$. Clearly, {\rm dim}\,{\rm ker}{\mathsf{Q}}_{0}^{+}={\rm dim}\,{\rm ker}\left[\begin{array}[]{cc}4&0\\ 0&0\end{array}\right]=1 and ${\rm dim}\,{\rm span}[{\mathsf{b}}_{1}]=1$, and so the dimension condition fails. On the other hand, our sufficient condition reads

[TABLE]

which is obviously satisfied.

6 Approximation hierarchies for ${\mathcal{C}}_{\Upsilon}$-copositivity

In general, checking copositivity of a matrix is an NP-hard problem, and hence solving a copositive optimization problem is also NP-hard. Therefore, to compute the semi-Lagrangian, we need to approximate them by so-called hierarchies, i.e., a sequence of conic optimization problems involving tractable cones ${\mathcal{K}}_{d}^{\star}$ such that ${\mathcal{K}}_{d}^{\star}\subset{\mathcal{K}}_{d+1}^{\star}\subset{\mathcal{C}}_{\Upsilon}^{\star}$ where $d$ is the level of the hierarchy, and ${\rm cl}(\bigcup_{d=0}^{\infty}{\mathcal{K}}_{d}^{\star})={\mathcal{C}}_{{\Upsilon}}^{\star}$. On the dual side, ${\mathcal{K}}_{d}$ are also tractable, ${\mathcal{K}}_{d+1}\subset{\mathcal{K}}_{d}$, and $\bigcap_{d=0}^{\infty}{\mathcal{K}}_{d}={\mathcal{C}}_{\Upsilon}$. For classical copositivity ${\mathcal{C}}_{{\mathbb{R}}^{n}_{+}}$, there are many options, for a concise survey see [8]. Many of these hierarchies involve linear  [11, 12] or psd constraints of matrices of order $n^{d+2}$, e.g. the seminal ones proposed in [24, 33]. One possibility would be the reduction of $\Upsilon$-copositivity via Schur complements as in Lemma 2.1 above, reducing this question to a combination of psd and classical copositivity conditions, which can be treated by these classical approximation hierarchies. However, the difficulty with this approach is the nonlinear dependence of the Schur complement ${\mathsf{R}}-{\mathsf{S}}^{\top}{\mathsf{H}}^{\dagger}{\mathsf{S}}$ on $(\mu,{\mathsf{u}})$. Therefore let us outline two alternative approaches for constructing tractable hierarchies in approximating the copositive relaxation, extending and adapting the classical approach.

SDP Hierarchy. One approach for computing the copositive relaxation is to use a hierarchy of SDP relaxations, extending the sum-of-squares idea in Parrilo’s work [33], which we sketch as below. Let $I=[1\!:\!p+1]$ and, for a matrix ${\mathsf{M}}={\mathsf{M}}^{\top}\in\mathbb{R}^{(p+n+1)\times(p+n+1)}$, define a quartic polynomial

[TABLE]

Note that ${\mathsf{M}}$ is $\Upsilon$-copositive if and only if $p_{\mathsf{M}}({\mathsf{y}})\geq 0$ for all ${\mathsf{y}}\in\mathbb{R}^{p+n+1}$. A sufficient condition for this is that the product $p_{\mathsf{M}}({\mathsf{y}}){\|{\mathsf{y}}\|}^{2d}=p_{\mathsf{M}}({\mathsf{y}})(\sum_{k}y_{k}^{2})^{d}$ is a sum-of-squares (s.o.s.) polynomial, which automatically guarantees nonnegativity of $p_{\mathsf{M}}$ over ${\mathbb{R}}^{p+n+1}$. Now it is natural to define

[TABLE]

Then, following the logic of classical hierarchies, it is not difficult to see that above properties hold, and that ${\mathcal{K}}_{d}^{\star}$ (and ${\mathcal{K}}_{d}$ itself) are tractable cones expressible by LMI conditions on matrices of order $n^{d+2}$. Thus, copositivity characterization of the semi-Lagrangian relaxation can be computed by using SDP hierarchies and polynomial optimization techniques [33]. Of course, LMIs on matrices of larger order pose a serious memory problem for algorithmic implementations even for moderate $d$ if $n$ is large.

However, in recent years, various techniques have been proposed to address this issue: one approach is to exploit special structures of the problem such as sparsity and symmetry [23, 22] to treat large scale polynomial problems. Other techniques include refined SDP hierarchies such as the SDP approximation proposed in [26] and the recently established bounded s.o.s. hierarchy [27].

On the other hand, it is worth noting that sometimes even the zero-level approximation in the hierarchy (18) can provide a much better bound as compared to the Lagrangian relaxation, as shown in the next example.

Example 6.1

(Zero-level approximation of copositive relaxations can beat the Lagrangian relaxation)* With the data from Example 3.1, recalling that the optimal value of this example is $z^{*}=0$, we have*

[TABLE]

Then the zero-level approximation for the copositive relaxation problem reads

[TABLE]

From the definition of ${\mathsf{M}}(u,\mu)$, we see that $p_{M(u,\mu)}$ is a s.o.s. polynomial if and only if $\mu\leq 0$ and

[TABLE]

is a s.o.s. polynomial. This shows that

[TABLE]

where

[TABLE]

Using the “solvesos” command in the Matlab toolbox YALMIP [29], one can verify that $\hat{p}_{u}$ is a s.o.s. polynomial if $u=1$. Thus, ${\rm val}\mathrm{(RP)}=0$ which agrees with the true optimal value $z^{*}=0$. On the other hand, as computed in Example 3.1, the Lagrangian relaxation yields a trivial lower bound $-\infty$.

LP Hierarchy. Another approach is to compute the copositive relaxation using linear optimization. While providing, in general, weaker bounds in comparing with SDP hierarchies, this approach is appealing because LP solvers suffer less from memory problems than state-of-art SDP solvers. To do this, consider a compact polyhedral base $K$ of the cone $\Upsilon$, i.e., ${\mathbb{R}}_{+}K=\Upsilon$, e.g. the polytope

[TABLE]

described by $m=p+2(n+1)$ affine-linear inequalities. By positive homogeneity, we observe that ${\mathsf{M}}$ is $\Upsilon$-copositive if and only if $q_{\mathsf{M}}({\mathsf{y}}):={\mathsf{y}}^{T}{\mathsf{M}}{\mathsf{y}}\geq 0$ for all ${\mathsf{y}}\in K$. Now Handelman’s theorem (for example see [25, Theorem 2.24]) ensures that any polynomial $f$ positive over such a polytope $K$ admits the representation $f=\sum_{{\bf\alpha}\in\mathbb{N}^{m}}c_{{\bf\alpha}}\prod_{j=1}^{m}h_{j}^{\alpha_{j}}$ for some scalars $c_{{\bf\alpha}}\geq 0$. Then one can construct a sequence of LP approximation by letting

[TABLE]

It is well known (see for example [25, Theorem 5.11]) that above ${\mathcal{K}}_{d}^{\star}$ can be expressed by linear inequality constraints.