New bounds for nonconvex quadratically constrained quadratic programming

TL;DR

This paper introduces new quadratic and cubic bounds for nonconvex quadratically constrained quadratic programming, connecting these bounds to semi-definite relaxations and comparing their effectiveness.

Contribution

It proposes novel quadratic and cubic bounds using affine and quadratic multipliers, linking them to existing relaxations and providing comparative analysis.

Findings

Most semi-definite relaxations are duals of quadratic bounds.

Cubic bounds outperform some traditional bounds in certain cases.

Comparison results show the effectiveness of proposed bounds.

Abstract

In this paper, we study some bounds for nonconvex quadratically constrained quadratic programs. We propose two types of bounds for quadratically constrained quadratic programs, quadratic and cubic bounds. For quadratic bounds, we use affine functions as Lagrange multipliers. We demonstrate that most semi-definite relaxations can be obtained as the dual of a quadratic bound. In addition, we study bounds obtained by changing the ground set. For cubic bounds, in addition to affine multipliers we employ quadratic functions. We provide a comparison between the proposed cubic bound and typical bounds for standard quadratic programs. Moreover, we report comparison results of a quadratic and a cubic bound for some non-convex quadratically constrained quadratic programs.

Tables

Bound hted	Shor relaxation	Bound (2)	Bound (2)	Bound (2)	Bound (2)	Bound (2)	Bound (3)
lb	-44.0945	-15.2676	-13.3647	-13.2518	-13.2294	-11.8969	-8.0008

Equations

$$\begin{array}[]{ll}&\min\ x^{T}Q_{0}x+2c_{0}^{T}x\\ &s.t.\ x^{T}Q_{i}x+2c_{i}^{T}x\leq b_{i},\ \ i=1,...,m\\ &\;\;\;\;\;\;\;\;\;Ax=d\\ &\;\;\;\;\;\;\;\;\;l\leq x\leq u,\end{array}$$

$$\max \ell$$

$$s.t.x^T{Q}_{0}x+2c_0^{T}x-\ell +\sum _{i=1}^m{\lambda }_i(x^T{Q}_{i}x+2c_i^{T}x-b_i)+\sum _{i=1}^p{\alpha }_i(x)(A_{i}x-d_i)-$$

$$\sum _{i=1}^n{\beta }_i(x)x_i+\sum _{i=1}^n{\gamma }_i(x)(x_i-1)\in {\mathcal{Q}}_{+}({\mathbb{R}}^n),$$

$${\alpha }_i\in \mathcal{A}({\mathbb{R}}^n),i=1,...,p$$

$${\lambda }_i\ge 0,{\beta }_i,{\gamma }_i\in {\mathcal{A}}_{+}(\mathcal{X}),i=1,...,n$$

$$\displaystyle\min\$$

$$\displaystyle s.t.\$$

$$X{A}_i^T=d_{i}x,i=1,...,p$$

$$Ax=d,$$

$$X\ge 0,$$

$$e{x}^T-X\ge 0,$$

$$X-e{x}^T-x{e}^T+e{e}^T\ge 0,$$

$$X-x{x}^T⪰0.$$

$$\displaystyle\max\$$

$$\displaystyle s.t.\$$

$$\sum _{i=1}^p{\alpha }_i(x)(A_{i}x-d_i)-\sum _{i=1}^n{\beta }_i(x)x_i+\sum _{i=1}^n{\gamma }_i(x)(x_i-1)\in {\mathcal{Q}}^N({\mathbb{R}}^n),$$

$${\lambda }_i\ge 0,{\alpha }_i\in \mathcal{A}({\mathbb{R}}^n),i=1,...,m,$$

$${\beta }_i,{\gamma }_i\in {\mathcal{A}}_{+}(\mathcal{X}),i=1,...,p.$$

$$\displaystyle\max\$$

$$\displaystyle s.t.\$$

$$\lambda \ge 0,{\alpha }_i\in \mathcal{A}({\mathbb{R}}^n)i=1,...,m$$

$$x^T{Q}_{i}x+2c_i^{T}x-b_i\le (or\ge )0,\forall x\in X,i=1,...,m$$

$${\mathcal{Q}}_{+}(B)=\{\left(\begin{array}{cc}{x}^T& 1\end{array}\right)Q\left(\begin{array}{c}x\\ 1\end{array}\right):Q\in K_B^{\ast }\}$$

$${\mathcal{Q}}_{+}(\mathcal{X})=\{\left(\begin{array}{cc}{x}^T& 1\end{array}\right)Q\left(\begin{array}{c}x\\ 1\end{array}\right):Q\in K_X^{\ast }\}$$

$$K_B=\mathrm{conv}\{z{z}^T:z\in {\mathbb{R}}_{+}^{n+1},z_i\le z_{n+1}i=1,...,n\}$$

$$K_{\mathcal{X}}=\mathrm{conv}\{z{z}^T:z\in {\mathbb{R}}_{+}^{n+1},\left(\begin{array}{cc}A& -b\end{array}\right)z=0,z_i\le z_{n+1}i=1,...,n\}.$$

$$K_F=\mathrm{conv}\{z{z}^T:z\in$$

$$j=1,...,m\}.$$

$$\begin{array}[]{ll}\min&x^{T}Q_{0}x+2c_{0}^{T}x\\ s.t.&A_{i}^{T}x=d_{i},\ \ \;\;\;\;i=1,...,p\\ &x_{i}\in\{0,1\}\;\;\;\;\;\;\;\;i\in\mathcal{I}\\ &0\leq x\leq e,\end{array}$$

$$\begin{array}[]{ll}\min&x^{T}Q_{0}x+2c_{0}^{T}x\\ s.t.&A_{i}^{T}x=d_{i},\ \ \;\;\;\;i=1,...,p\\ &x_{i}(1-x_{i})\leq 0\;\;\;\;\;\;\;\;i\in\mathcal{I}\\ &0\leq x\leq e.\end{array}$$

$$q(x)+\sum _{i=1}^n{\alpha }_i(x,s)(x_i+s_i-1)\in {\mathcal{Q}}_{+}({\mathbb{R}}_{+}^{2n}),$$

$${\alpha }_i(x,s)\in \mathcal{A}({\mathbb{R}}_{+}^{2n}),i=1,...,n$$

$$\left(\begin{array}{cc}I& -R_{i}x\\ -x^T{R}_i^T& -2c_i^{T}x+b_i,\end{array}\right)⪰0.$$

$$\min f^{T}x$$

$$s.t.\left(\begin{array}{cc}I& -R_{i}x\\ -x^T{R}_i^T& -2c_i^{T}x+b_i,\end{array}\right)⪰0.$$

$$\displaystyle\min\$$

$$\displaystyle\ s.t.\$$

$$X{A}_i^T=d_{i}x,i=1,...,p$$

$$Ax=d,$$

$$X\ge 0,X-x{x}^T⪰0,$$

$$e{x}^T-X\ge 0,$$

$$X-e{x}^T-x{e}^T+e{e}^T\ge 0,$$

$$\left(\begin{array}{cc}{x}_{k}I& R_{i}X{e}_k\\ (R_{i}X{e}_k{)}^T& -2c_i^{T}X{e}_k+b_i{e}_k^{T}x\end{array}\right)⪰0,i=1,...,m_c,k=1,...,n$$

$$\left(\begin{array}{cc}(1-x_k)I& -R_{i}X{e}_k+R_{i}x\\ (-R_{i}X{e}_k+R_{i}x{)}^T& 2c_i^{T}X{e}_k-(2c_i^T+b_i{e}_k^T)x+b_k\end{array}\right)⪰0,i=1,...,m_c,k=1,...,n$$

Full text

1 Introduction

We consider the following quadratically constrained quadratic programming, QCQP,

[TABLE]

where $x\in\mathbb{R}^{n}$ is the vector of decision variables, $Q_{i}$ ($i=0,1,...,m$) are $n\times n$ real symmetric matrices, $A$ is a $p\times n$ real matrix, $c_{i}$ ($i=0,1,...,m$) and $d$ are vectors in $\mathbb{R}^{n}$ and $\mathbb{R}^{p}$, respectively, and $b_{i}$ ($i=1,...,m$) are real scalars. We assume that $-\infty<l_{i}\leq u_{i}<\infty$ for $i=1,...,n$. Without loss of generality, we may assume that $l=0$ and $u=e$, where $e$ represents vector of ones in $\mathbb{R}^{n}$. We remark that general QCQPs with bounded feasible set can be formulated as (QCQP).

QCQP is a fundamental problem in optimization theory and practice. QCQPs arise in many applications including economic equilibria, facility location and circle packing problems [3, 23, 26]. Furthermore, most combinatorial optimization problems including max-cut problem and clique problem can be casted as QCQPs [9, 4]. In addition to the aforementioned problem, Madani et al. showed that any polynomial optimization problem can be casted as a QCQP [15]. When the matrices $Q_{i}$ ($i=0,1,...,m$) are positive semi-definite (QCQP) will be a convex optimization problem, and it is polynomially solvable. Nevertheless, as QCQPs include a wide range of NP-hard optimization problems, QCQP is NP-hard [25].

A typical class of optimization methods for handling QCQPs is branch-and-bound method. In this approach, the main problem is divided to some subproblems, which are called nodes. At each node, a lower bound is computed by a relaxation or a bound. In general, the generated lower bound determines a node will be fathomed or branched. The effectiveness of a branch-and-bound method rests mainly on the tightness of generated lower bounds and their computational time. Most relaxation approaches and bounds for QCQPs are mainly based on the reformulation-linearization technique (RLT), convex relaxations and semidefinite relaxations [24, 21, 12]. The most effective relaxation method is semi-definite relaxation (SDR) [3, 10]. Due to the efficiency of this approach, many SDRs have been proposed; See [3, 28] for review and comparisons.

Recently, Zamani has proposed a new dual for linearly constrained quadratic programming [27]. He considers affine functions as Lagrange multipliers. In this paper, similar to his method, we present two types of bounds for QCQPs, quadratic and cubic bounds. For quadratic bounds, we employ affine functions as Lagrange multipliers. We illustrate that most SDRs can be interpreted as the dual of a quadratic bound. In addition, we introduce some bounds which are obtained by changing the ground set.

For cubic bounds, we apply quadratic functions as Lagrange multipliers. We give some conditions under which the proposed bound is exact. We demonstrate that the cubic bound is equivalent the bound obtained by Parrilo hierarchy for standard quadratic programs.

The paper is organized as follows. After reviewing our notations, in Section 2 we introduce the quadratic bounds. Section 3 is devoted to cubic bounds. In Section 4, we illustrate the effectiveness of a quadratic and a cubic bound by presenting its numerical performance on some QCQPs.

1.1 Notation

The following notation is used throughout the paper. The $n$-dimensional Euclidean space is denoted by $\mathbb{R}^{n}$. Let $A_{i}$ stand for the $i^{th}$ row of matrix $A$. Vectors are considered to be column vectors and $T$ denotes transposition operation. We employ $e_{i}$ to represent the $i^{th}$ unit vector, and vector $e$ stands for vector of ones. For symmetric matrices $A$ and $B$, we use notation $A\succeq B$ to denote $A-B$ is positive semidefinite. The inner product $A$ and $B$ is defined and denoted as $A\bullet B=trace(AB)$. A symmetric $n\times n$ matrix $Q$ is called copositive if bilinear form $x^{T}Qx$ is non-negative on non-negative orthant. For $x\in\mathbb{R}^{n}$, we denote by $\operatorname{diag}(x)$ the diagonal matrix with $\operatorname{diag}(x)_{ii}=x_{i}$. Moreover, for $n\times n$ matrix $Q$, $\operatorname{Diag}(Q)$ denotes a column vector with $\operatorname{Diag}(Q)_{i}=Q_{ii}$.

For a set $\mathcal{X}\subseteq\mathbb{R}^{n}$, we use the notations $\operatorname{int}(\mathcal{X})$ and $\operatorname{conv}(\mathcal{X})$ for the interior and the convex hull of $\mathcal{X}$, respectively. $\mathbb{R}_{+}^{n}$ denotes non-negative orthant. We use $B$ to represent box $[0,1]^{n}$. The dual cone of $K$ is denoted by $K^{*}=\{y:y^{T}x\geq 0,\ \forall x\in K\}$.

We use $\mathcal{A}(\mathbb{R}^{n})$ and $\mathcal{Q}(\mathbb{R}^{n})$ to represent affine and quadratic functions on $\mathbb{R}^{n}$. We denote non-negative affine and quadratic functions on $\mathcal{X}\subseteq\mathbb{R}^{n}$ by $\mathcal{A}_{+}(\mathcal{X})$ and $\mathcal{Q}_{+}(\mathcal{X})$, respectively, that is $\mathcal{A}_{+}(\mathcal{X})=\{\alpha\in\mathcal{A}(\mathbb{R}^{n}):\alpha(x)\geq 0\ \forall x\in\mathcal{X}\}$ and $\mathcal{Q}_{+}(\mathcal{X})=\{q\in\mathcal{Q}(\mathbb{R}^{n}):q(x)\geq 0\ \forall x\in\mathcal{X}\}$. For a quadratic function $q(x)=x^{T}Qx+2c^{T}x+b$, we denote the matrix representation of $q$ by $\mathcal{M}(q)=\begin{pmatrix}Q&c\\ c^{T}&b\end{pmatrix}$.

2 Quadratic bounds

In this section, we propose some quadratic bounds for QCQP. Let $\mathcal{X}=\{x\in\mathbb{R}^{n}:x\in B,Ax=d\}$ and $F=\{x\in\mathbb{R}^{n}:x\in\mathcal{X},x^{T}Q_{i}x+2c_{i}^{T}x\leq b_{i},i=1,...,m\}$. Because $\mathcal{A}_{+}(\mathcal{X})$ is a polyhedral set, it follows that the representation of $\mathcal{A}_{+}(\mathcal{X})$ in $\mathbb{R}^{n+1}$ is a polyhedral cone [16]. We propose the following problem as a new quadratic bound for (QCQP),

[TABLE]

The above problem can be regarded as a Lagrangian dual for (QCQP) for which the dual variables corresponding to linear constraints are replaced by affine functions. We remark that, due to the non-homogeneous Farkas’ Lemma, $\alpha(x)=f^{T}x+g$ belong to $\mathcal{A}_{+}(\mathcal{X})$ if and only if there exist $\lambda\in\mathbb{R}^{p}$ and $\mu\in\mathbb{R}_{+}^{n}$ with $f\geq A^{T}\lambda-\mu$ and $g\geq d^{T}\lambda+e^{T}\mu$. Note that the quadratic function $q(x)=x^{T}Qx+2c^{T}c+b$ is non-negative on $\mathbb{R}^{n}$ if and only if matrix $\mathcal{M}(q)$ is positive semi-definite, and accordingly problem (2) can be formulated as a semi-definite program, which has $O(n^{2})$ variables.

One crucial question regarding this bound is well-definedness. In next proposition, we prove that problem (2) is feasible and generates a finite lower bound for (QCQP).

Proposition 2.1**.**

Let (QCQP) have a feasible point. Then problem (2) gives a finite lower bound.

Proof.

Similar to the proof of Proposition 2 in [27], it is shown that there exist $\gamma_{i}\in\mathcal{A}_{+}(\mathcal{X})$ for $i=1,...,n$ such that $x^{T}Q_{0}x+2c^{T}_{0}x+\sum_{i=1}^{n}\gamma_{i}(x)(x_{i}-1)$ is strictly convex. So for suitable choice of $\ell$, we have $x^{T}Q_{0}x+2c^{T}_{0}x+\sum_{i=1}^{k}\gamma_{i}(x)(x_{i}-1)-\ell\in\mathcal{Q}_{+}(\mathbb{R}^{n})$, which shows the feasibility of (2). The first constraint of (2) implies that the optimal value of (2) is a lower bound for (QCQP). ∎

The proof of above theorem reveals that problem (2) is feasible for each quadratic function. In fact, the problem is strongly feasible. The conic optimization problem is called strongly feasible if it is feasible and remains feasible for all sufficiently small perturbations of right side of linear constraints [19]. As problem (2) is convex, it is natural to ask about the dual thereof. The dual of (2) can be written as

[TABLE]

We refer the reader for the details of computation of (2) to [27]. In problem (2), we did not write redundant constraint $x\in B$ [22]. Problem (2) is a well-known Shor relaxation with partial first-level RLT [1, 3]. Anstreicher proposed SDR (2) as a combination of RLT and Shor relaxation. He showed that SDR (2) can generate bounds tighter than either technique. Bao et al. established that SDR (2) and doubly non-negative relaxation provide the same bound. The doubly non-negative relaxation is similar to problem (2), but the constraint $XA_{i}^{T}=d_{i}x$ is replaced by $X\bullet A_{i}^{T}A_{i}=d_{i}^{2}$, $i=1,...,m$. Note that as (2) is strongly feasible, strong duality holds [19], and consequently problems (2) and (2) are equivalent.

Since the ground set of (QCQP) is not $\mathbb{R}^{n}$, the bound may be improved if we replace $\mathcal{Q}_{+}(\mathbb{R}^{n})$ by other sets. Bomze took advantage of this idea and has proposed some results about global optimality conditions for QCQPs [5]. As the feasible set of (QCQP) is subset of positive orthant, one replacement for $\mathcal{Q}_{+}(\mathbb{R}^{n})$ can be quadratic functions with non-negative coefficients. In this case, we have the following bound

[TABLE]

where $\mathcal{Q}^{N}(\mathbb{R}^{n})$ denotes quadratic functions with non-negative coefficients. The above problem can be formulated as a linear program. It can be shown that (2) is the dual of a linear RLT [22].

Another interesting substitute for $\mathcal{Q}_{+}(\mathbb{R}^{n})$ is non-negative quadratic functions on $B$. In this case, the following program provides a bound

[TABLE]

Since for each $q\in\mathcal{Q}(\mathbb{R}^{n})$, there exists $\ell$ with $q-\ell\in\mathcal{Q}_{+}(B)$, problem (2) is always feasible. Needless to say, bound (2) dominates all the above-mentioned bounds. Nevertheless, this bound is not necessarily exact for general QCQPs. A bound or relaxation is said to be exact if it provides a bound equal to the optimal value of main problem. Next theorem provides some sufficient conditions for exactness.

Theorem 2.2**.**

Bound (2) is exact if

[TABLE]

Proof.

First we prove the case that there does not exist any quadratic constraint. By Lemma 4 in [8],

[TABLE]

where

[TABLE]

Remark that $\mathcal{M}(\mathcal{Q}_{+}(B))^{*}=K_{B}$ and $\mathcal{M}(\mathcal{Q}_{+}(\mathcal{X}))^{*}=K_{\mathcal{X}}$. We show that $\mathcal{Q}_{+}(\mathcal{X})=\mathcal{Q}_{+}(B)+\{\sum_{i=1}^{p}\alpha_{i}(x)(A_{i}x-d_{i}):\alpha_{i}\in\mathcal{A}(\mathbb{R}^{n})\}$. The inclusion $\supseteq$ is trivial. We prove the inclusion $\subseteq$ by contradiction. Let $q(x)=x^{T}Qx+2c^{T}x+c_{0}\in\mathcal{Q}_{+}(\mathcal{X})$ while $q\notin\mathcal{Q}_{+}(B)+\{\sum_{i=1}^{p}\alpha_{i}(x)(A_{i}x-d_{i}):\alpha_{i}\in\mathcal{A}(\mathbb{R}^{n})\}$. By separation theorem, there is $O\in\mathcal{M}(\mathcal{Q}_{+}(B))^{*}+\mathcal{M}(\{\sum_{i=1}^{p}\alpha_{i}(x)(A_{i}x-d_{i}):\alpha_{i}\in\mathcal{A}(\mathbb{R}^{n})\})^{*}$ with $\mathcal{M}(q)\bullet O=-1$. As $\mathcal{M}(\mathcal{Q}_{+}(B))^{*}=K_{B}$, we have $O=\sum_{k=1}^{l}z^{k}(z^{k})^{T}$, where $z^{k}\in\{\mathbb{R}^{n+1}_{+},z_{i}\leq z_{n+1}\ i=1,...,n\}$ ($k=1,...,l$). As $(a_{i}x-b)(a_{i}x-b)$ and $-(a_{i}x-b)(a_{i}x-b)$ are members of $\{\sum_{i=1}^{p}\alpha_{i}(x)(A_{i}x-d_{i}):\alpha_{i}\in\mathcal{A}(\mathbb{R}^{n})\}$ for $i=1,...,p$, we have $O\in K_{\mathcal{X}}$. This implies that $O\in\mathcal{M}(\mathcal{Q}_{+}(\mathcal{X}))^{*}$. Thus, we have $\mathcal{M}(q)\bullet O\geq 0$ which contradicts the assumption $\mathcal{M}(q)\bullet O=-1$.

Now we consider the case that quadratic constraints exist. If $x^{T}Q_{i}x+2c^{T}_{i}x\leq b_{i}$ for each $x\in\mathcal{X}$ and $i=1,...,m$, the quadratic constraints are redundant and theorem follows from the first part.

For the case that $x^{T}Q_{i}x+2c^{T}_{i}x\geq b_{i}$ for each $x\in\mathcal{X}$ and $i=1,...,m$, we establish that the dual cones corresponding to $\mathcal{M}(\mathcal{Q}_{+}(F))$ and $\mathcal{M}(\mathcal{Q}_{+}(A)+\{\sum_{i=1}^{m}\lambda_{i}(x^{T}Q_{i}x+2c^{T}_{i}x-b_{i}):\lambda\leq 0\})$ are the same. The inclusion $\mathcal{M}(\mathcal{Q}_{+}(F))^{*}\subseteq\mathcal{M}(\mathcal{Q}_{+}(A)+\{\sum_{i=1}^{m}\lambda_{i}(x^{T}Q_{i}x+2c^{T}_{i}x-b_{i}):\lambda\leq 0\})^{*}$ is immediate. We show the reverse inclusion. Suppose that $O\in\mathcal{M}(\mathcal{Q}_{+}(A)+\{\sum_{i=1}^{m}\lambda_{i}(x^{T}Q_{i}x+2c^{T}_{i}x-b_{i}):\lambda\leq 0\})^{*}$. By the representation of $\mathcal{Q}_{+}(A)$, we have $O=\sum_{k=1}^{l}z^{k}(z^{k})^{T}$, where $z^{k}\in\{\mathbb{R}^{n+1}_{+},\begin{pmatrix}A&-d\end{pmatrix}z=0,z_{i}\leq z_{n+1}\ i=1,...,n\}$ ($k=1,...,l$). By the assumption $\begin{pmatrix}Q_{i}&c_{i}\\ c_{i}^{T}&b_{i}\end{pmatrix}\bullet z_{k}z_{k}^{T}\geq 0$ ($i=1,...,m,k=1,...,l$), we have $z_{i}^{T}Oz_{i}=0$. Suppose that

[TABLE]

As $\mathcal{Q}_{+}(F)=\{\begin{pmatrix}x^{T}&1\end{pmatrix}Q\begin{pmatrix}x\\ 1\end{pmatrix}:Q\in K_{F}^{*}\}$, we have $O\in\mathcal{M}(\mathcal{Q}_{+}(F))^{*}$, which completes the proof. ∎

It is worth mentioning that the Theorem 2.2 can be proved by using strong duality for conic programs and Proposition 6 in [3], but here we present a new proof. Next proposition states that bound (2) is exact for linearly constrained quadratic programs with binary variables.

Proposition 2.3**.**

Bound (2) is exact for linearly constrained quadratic programs with binary variables.

Proof.

Consider the problem

[TABLE]

where index set $\mathcal{I}\subseteq\{1,...,n\}$ denotes binary variables. This problem can be formulated as

[TABLE]

As all conditions of Theorem (2.2) holds for the above problem, bound (2) is exact for linearly constrained quadratic programs with binary variables. ∎

By Theorem 2.6 in [7] and strong duality for conic programs $q\in\mathcal{A}_{+}(B)$ if and only if the following system has a solution

[TABLE]

where slack variable $s\in\mathbb{R}^{n}$. A quadratic function $q(x)=x^{T}Qx+2c^{T}x+b\in\mathcal{Q}_{+}(\mathbb{R}_{+}^{n})$ if and only if matrix $\mathcal{M}(q)$ is copositive [5]. So bound (2) can be casted as a copostive program. Copostive programs are intractable in general. In fact, they are NP-hard. Nonetheles, there are efficient methods which approximate copositive cone [6, 18].

It is well-known Shor relaxation is the dual of (QCQP) when affine multipliers are constant functions. In addition, we have shown before, the dual of (2) is Shor relaxation with partial first-level RLT. It is may be of interest to know whether other SDRs can be also obtained in this manner. In the sequel, we will show that some SDRs can be obtained as the dual of bounds in the form of (2) by a suitable choice of affine multipliers or adding some valid cuts.

Let $m_{c}=\{i:Q_{i}\succeq 0,Q_{i}\neq 0\}$. In the rest of the section, we make the assumption that for each $i\in m_{c}$ there exists $\bar{x}^{i}\in\mathbb{R}^{n}$ such that $(\bar{x}^{i})^{T}Q_{i}\bar{x}^{i}+2c^{T}\bar{x}^{i}<b_{i}$. Due to the semi-positiveness of $Q_{i}$ ($i\in m_{c}$), there exists matrix $R_{i}$ with $Q_{i}=R_{i}^{T}R_{i}$. By Schur Complement Lemma, $x^{T}Q_{i}x+2c_{i}^{T}x\leq b_{i}$ is equivalent to

[TABLE]

So affine function $\alpha(x)=f^{T}x+g$ is non-negative on $L_{i}=\{x\in\mathbb{R}^{n}:x^{T}Q_{i}x+2c_{i}^{T}x\leq b_{i}\}$ if and only if the optimal value of the following semi-definite program is greater than or equal to $-g$.

[TABLE]

We remark that problem (2) can be reformulated as a second-order cone program. Recently, Zheng et al. proposed some SDRs for QCQPs [28]. In fact, they introduced a unified framework for generating convex relaxations for QCQPs. They propose the following SDR for (QCQP)

[TABLE]

and they called it SDP relaxation with rank-2 second-order cone valid inequalities. Note that the above SDR is obtained by adding the last two constraints of (2) to Shor relaxation with partial first-level RLT. We demonstrate that SDR (2) is the dual of the following bound

[TABLE]

It is easily seen (2) is a bound for (QCQP). For convenience, to show bound (2) is the dual of (2) we consider the QCQP

[TABLE]

which has a convex quadratic constraint and a linear inequality constraint. Bound (2) for problem (8) is formulated as

[TABLE]

As $\operatorname{int}(L_{1})\neq\emptyset$, the strong duality holds for problem (2). Accordingly, $f^{T}x+g\in\mathcal{A}_{+}(L_{1})$ is equivalent that the optimal value of the following semi-definite program is greater or equal to $-g$,

[TABLE]

Hence, problem (9) is reformulated as

[TABLE]

By a little algebra, the dual of the above problem is formulated as

[TABLE]

which clarifies the point that bound (2) is the dual of (2). Since (2) is strongly feasible, strong duality also holds.

A typical method to tighten the relaxation gap is adding valid cuts. Zheng et al. introduced a class of quadratic valid cuts for QCQP and they proposed a new SDR by using these valid cuts [28]. Their method generate a quadratic valid cut as follows. Let $F\subseteq\Omega$. Suppose that $u\in\operatorname{int}(\mathbb{R}^{n}_{+})$ and $0<u_{\Omega}=\{\max\ u^{T}x:\ x\in\Omega\}$. They showed that for $S\succeq 0$, the convex quadratic inequality $x^{T}Sx-u_{\Omega}\operatorname{Diag}(S)^{T}\operatorname{diag}(u)^{-1}x\leq 0$ is valid for (QCQP), see Proposition 3 in [28]. We remark that the set of generated cuts by this method forms a convex cone in $\mathcal{Q}({\mathbb{R}^{n}})$.

By the above discussion, one can extend bound (2) as follows,

[TABLE]

which is a non-convex optimization problem with infinite constraints and variables. As mentioned above, the set of valid cuts is a convex cone, so the above bound can be formulated as semi-definite program

[TABLE]

The dual of bound (2) is formulated as

[TABLE]

which is the SDR proposed in [28]; See $SDP_{\alpha_{u}}$. Since (2) is also strongly feasible, we have strong duality. Here we just investigate some well-known SDRs and show that they can be interpreted as the dual of a bound in the form (2). However, by a similar argument one can show that most SDRs can be obtained as the dual of a bound in the form of (2).

We conclude the section by mentioning some points. As the dual of the proposed bounds are well-known SDRs, we have just reinvented the wheel. Of course, this statement is correct, but viewing SDRs from this aspect can supply us with more tools for analyzing a SDR method. For instance, one can extend bound (2) as follows

[TABLE]

where $V=\{x\in\mathcal{X}:x^{T}Q_{i}x+2c_{i}^{T}x\leq b_{i},\ i\in m_{c}\}$. As $\mathcal{A}_{+}(\mathcal{X})\cup_{i\in m_{c}}\mathcal{A}_{+}(L_{i})\subseteq\mathcal{A}_{+}(V)$, bound (2) dominates (2). Therefore, the dual of (2) leads to a SDR which dominates (2). Here, it is assumed that $\operatorname{int}(\cap_{i\in m_{c}}L_{i})\cap\mathcal{X}\neq\emptyset$. In the same line, we can formulate the following bound which dominates (2)

[TABLE]

where $W=\{x\in V:x^{T}Sx-u_{\Omega}\operatorname{Diag}(S)^{T}\operatorname{diag}(u)^{-1}x\leq 0,\ \forall S\succeq 0\}$. Note that if there exists $\bar{x}\in\mathcal{X}$ with $\bar{x}^{T}S\bar{x}-u_{\Omega}\operatorname{Diag}(S)^{T}\operatorname{diag}(u)^{-1}\bar{x}<0$ for $S\succeq 0$ and $\bar{x}^{T}Q_{i}\bar{x}+2c_{i}^{T}\bar{x}<b_{i}$ ($i\in m_{c}$), then $f^{T}x+g\in\mathcal{A}_{+}(W)$ is equivalent to the consistency of the system

[TABLE]

Thus, bound (2) is reformulated as a semi-definite program, and consequently its dual gives a SDR which dominates (2).

Another point about the proposed bounds is that they not only provide a lower bound, but also give a convex underestimator. The given convex underestimator can be employed in optimization methods for generating a solution.

It is well-known when an optimal solution of a SDR has rank one the SDR is exact [14]. The next proposition gives necessary and sufficient conditions for exactness. For convenience to state the proposition, we consider bound (2). Let $F_{opt}$ denote the optimal solution set of (QCQP).

Proposition 2.4**.**

Bound (2) is exact if and only if there exists feasible point $\bar{\lambda}$, $\bar{\alpha}_{i}$ ($i=1,...,p$), $\bar{\beta}_{i},\bar{\gamma}_{i}$ ($i=1,...,n$) and $\bar{\ell}$ with

[TABLE]

Proof.

Let (2) be exact and suppose that $\bar{\lambda}$, $\bar{\alpha}_{i}$ ($i=1,...,p$), $\bar{\beta}_{i},\bar{\gamma}_{i}$ ($i=1,...,n$) and $\bar{\ell}$ is an optimal solution. As the bound is exact, we have $\bar{x}^{T}Q_{0}\bar{x}+2c^{T}_{0}\bar{x}=\bar{\ell}$ for $\bar{x}\in F_{opt}$. In the light of $q(x)=x^{T}Q_{0}x+2c^{T}_{0}x-\bar{\ell}+\sum_{i=1}^{k}\bar{\lambda}_{i}(x^{T}Q_{i}x+2c^{T}_{i}x-b_{i})+\sum_{i=1}^{k}\bar{\alpha}_{i}(x)(A_{i}x-d_{i})-\sum_{i=1}^{k}\bar{\beta}_{i}(x)x_{i}+\sum_{i=1}^{k}\bar{\gamma}_{i}(x)(x_{i}-1)\in\mathcal{Q}_{+}(\mathbb{R}^{n})$ and $q(\bar{x})=0$, we have

[TABLE]

which completes the if part. The only-if part is immediate. ∎

It is worth mentioning that as strong duality holds for all proposed bounds, exactness of SDRs and bounds are equivalent. Moreover, bound (2) or SDR (2) are exact for general QCQP if and only if $n=2$, see [2] for more details.

3 Cubic bounds

In this section, we propose cubic bounds for QCQP. So far, we use affine functions as dual variables. The most important point for applying other functions is that the obtained problem should be tractable.

Due to the structure of (QCQP), one may consider the following convex cones for dual variables,

$\mathcal{Q}_{+}^{c}(\mathcal{X})$: non-negative convex quadratic functions on $\mathcal{X}$, 2. 2.

$\mathcal{Q}^{N}(\mathbb{R}^{n})$: quadratic functions with non-negative coefficients.

Both above-mentioned cones have non-empty interior and verifying the membership of a given quadratic function is tractable. Verifying $q\in\mathcal{C}^{N}(\mathbb{R}^{n})$ is straightforward. By alternative theorem [16], $q(x)=x^{T}\hat{Q}x+2\hat{c}^{T}c+\hat{c}_{0}$ belongs to $\mathcal{Q}_{+}^{c}(\mathcal{X})$ if and only if there exist $\lambda\in\mathbb{R}^{p}$ and $\mu,\nu_{2}\in\mathbb{R}^{n}_{+}$ with

[TABLE]

By employing quadratic functions as dual variables, we are faced with checking non-negativity of a cubic function. Of course, a cubic function may not be non-negative on $\mathbb{R}^{n}$, unless it is quadratic. So it appears our effort by substituting some class of quadratic functions instead of affine functions was in vain. Nevertheless, checking non-negativity of some class of cubic functions are tractable on non-negative orthant. For instance, one may consider the following sets of cubic functions,

$\mathcal{C}_{+}^{c}(\mathbb{R}_{+}^{n})$: non-negative convex cubic functions on $\mathbb{R}_{+}^{n}$, 2. 2.

$\mathcal{C}^{N}(\mathbb{R}^{n})$: cubic functions with non-negative coefficients,

Both sets are convex cones with non-empty interior. In addition, to check a cubic function belongs to these cones are tractable. Let $q(x)=Tx^{3}+xQx+cx+c_{0}$ be a cubic function, where $T$ is a symmetric tensor of order 3. Verifying $q\in\mathcal{C}^{N}(\mathbb{R}^{n})$ is straightforward. To check $q\in\mathcal{C}_{+}^{c}(\mathbb{R}_{+}^{n})$, we need first to impose the following conditions

[TABLE]

which guarantees convexity of $q$ on $\mathbb{R}_{+}^{n}$. As $q$ is convex, its optimal value can be obtained by primal interior point methods. As a result, membership verification is tractable in this case, but cannot be checked explicitly by some linear inequalities.

Another replacement for $\mathcal{C}_{+}^{c}(\mathbb{R}_{+}^{n})$ or $\mathcal{C}^{N}(\mathbb{R}^{n})$ may be the set of quadratically Sum-of-Squares. We call a cubic function $q(x)=Tx^{3}+x^{T}Qx+c^{T}x+c_{0}$ quadratically Sum-of-Squares if $T(x^{(2)})^{3}+(x^{(2)})^{T}Q(x^{(2)})+c(x^{(2)})+c_{0}$ is Sum-of-Squares, where $x^{(2)}=(x_{1}^{2},...,x_{n}^{2})$. Note that one can check whether a polynomial is Sum-of-Squares using semidefinite programming [11].

By the above discussion, we propose the following bound for (QCQP)

[TABLE]

By the above discussion, problem (3) can be formulated as a semi-definite program and it has $O(n^{3})$ variables. By Proposition 2.1, bound (3) is always finite and generates a lower bound greater or equal to that of (2).

One may wonder how the bound given by (3) can be improved. The straightforward method for tightening can be enlargement of feasible set. In problem (3), we have linear and quadratic function variables. One can adopt methods in Section 2 to tighten bound (3). The following proposition gives necessary and sufficient conditions under which bound (3) is exact.

Proposition 3.1**.**

Bound (3) is exact if and only if there exists feasible point $\bar{\lambda}_{i}$ ($i=1,...,m$), $\bar{\alpha}_{i}$ ($i=1,...,p$), $\bar{\beta}_{i},\bar{\gamma}_{i}$ ($i=1,...,n$), $\bar{\ell}$ and $\bar{\kappa}$ with

[TABLE]

Proof.

Similar to Proposition 2.4 is proved. ∎

In the same line for problem (3), one could consider quadratics or linear multipliers for which optimal value of (3) are non-negative. In fact, one may consider $q(x)=x^{T}\hat{Q}x+2\hat{c}^{T}x+\hat{c}_{0}$ eligible if the following system has a solution

[TABLE]

Thus, we obtain a hierarchy for tackling (QCQP). This hierarchy is increasing and each problem is formulated as a semi-definite program. If we consider problem (3) as a first problem of hierarchy, the $k^{th}$ problem contains $O(n^{k+2})$ variables. The most important inquiry concerning this method is its convergence in finite steps. In addition, if it is convergent in finite steps, what the order of $k$ for which we have convergence. As the subject of the paper is quadratic and cubic bounds, we leave these questions for further research.

The following example demonstrate that (3) could generate a bound tighter than the proposed bounds in Section 2.

Example 3.2**.**

Consider the nonconvex QCQP,

[TABLE]

The problem has two convex quadratic constraints, one nonconvex quadratic constraint and nine linear constraints, with the optimal value is $-8.0008$ and the optimal solution $\begin{pmatrix}0.4203,&0.4942,&0.7956,&0\end{pmatrix}$. We set $u=\begin{pmatrix}1,&2,&2,&1\end{pmatrix}^{T}$ and $u_{\Omega}=max\{u^{T}x:\ x\in F\}=3.9145$. The performance of the bounds are listed in Table 1, which lb denotes the generated lower bound.

As seen bound (3) is exact for the example.

In the rest of the section, we investigate the relationship between bound (3) and the conventional lower bounds for QCQPs. Due to the computational burdensome, cubic bounds are not commonplace and they have been applied just for some types of QCQPs such as standard quadratic programs. Of course, we can provide a comparison between bound (3) and general polynomial optimization methods, including Lasserre hierarchy [11], with $O(n^{3})$ variables, but we prefer bounds tailored for QCQPs.

Consider the standard quadratic program,

[TABLE]

It is well-known that (StQP) is solvable in polynomial time provided $Q$ is either positive semi-definite or negative semi-definite on standard simplex. In general, however, (StQP) is NP-hard [6]. Suppose that $\Delta$ denote the standard simplex.

Let $\ell_{Q}$ denote the optimal value of (StQP). We remark that optimizing a quadratic function on standard simplex can be formulated as (StQP). This is resulted from the fact that for each $x\in\Delta$, we have $x^{T}Qx+2c^{T}x=x^{T}(Q+ec^{T}+ce^{T})x$.

One effective method for handling (StQP) is Parrilo hierarchy [6]. In this method, for $r=0,1,...$ the following problem gives a lower bound

[TABLE]

where $\mathcal{P}^{r}=\{A:(􏰄\sum_{i=1}^{n}x_{i}^{2})(􏰄􏰄\sum_{i=1}^{n}􏰄\sum_{j=1}^{n}x_{i}^{2}A_{ij}x_{j}^{2})\in\Sigma[x]\}$ and $\Sigma[x]$ denotes set of all sum of square polynomials. It is well-known for sufficiently large $r$, $p^{r}_{Q}$ is equal to the optimal value of (StQP) [6]. In addition, the number of variables of (15) is of $O(n^{r+2})$ [18].

Bound (3) is formulated as follows for (StQP),

[TABLE]

As mentioned earlier, the number of variables of (3) is of $O(n^{3})$. So, one may wonder what is the relationship between $p^{1}_{Q}$ and the optimal value of $\eqref{BSQ}$ The following theorem says these bounds are equivalent. Before we get to the proof, let us mention some points. It is shown in [6, 18], the symmetric matrix $B\in\mathcal{P}^{1}$ if and only if there exist symmetric matrices $K^{(1)},...,K^{(n)}$ with

[TABLE]

Let convex quadratic function $q(x)=x^{T}Sx+2c^{T}x+c_{0}$ be nonnegative on $\Delta$. It is easily seen that $(􏰄\sum_{i=1}^{n}x_{i})^{2}q(􏰄\sum_{i=1}^{n}x_{i})^{-1}x)$ is homogeneous polynomial of degree two. So for some symmetric matrix $Q$, we have $(􏰄\sum_{i=1}^{n}x_{i})^{2}q(􏰄\sum_{i=1}^{n}x_{i})^{-1}x)=x^{T}Qx$. As $q\in\mathcal{Q}_{+}^{c}(\Delta)$, there exist nonnegative multipliers $\lambda_{i}$ ($i=1,...,n$) and $\lambda_{n+1}$ with

[TABLE]

By the replacement of $x$ with $(\sum_{i=1}^{n}x_{i})^{-1}x$ and multiplication of $(􏰄\sum_{i=1}^{n}x_{i})^{2}$, it is readily seen that $Q$ can be represented as a summation of a positive semi-definite matrix and a nonnegative matrix.

Theorem 3.3**.**

Bounds $p^{1}_{Q}$ and (3) are equivalent.

Proof.

First, we show that the optimal value of $\eqref{BSQ}$ is less than or equal to $p^{1}_{Q}$. Let $\bar{\alpha}_{i}(x)=x^{T}S_{i}x+2c^{T}_{i}x+g_{i}$, $i=1,...,n+1$, $\bar{\kappa}$ and $\bar{\ell}$ be an optimal solution of $\eqref{BSQ}$. (Without loss of generality, it is assumed $\eqref{BSQ}$ attains its optimal solution.) As $\bar{\kappa}$ is nonnegative on standard simplex, $(e^{T}x)^{3}\bar{\kappa}((e^{T}x)^{-1}x)$ is homogeneous polynomial of degree three with nonnegative coefficients. Thus, for nonnegative symmetric matrix $K_{i}$, $i=1,...,n$, we have $(e^{T}x)^{3}\bar{\kappa}((e^{T}x)^{-1}x)=\sum_{i=1}^{n}x_{i}(x^{T}K_{i}x)$. Furthermore, $(e^{T}x)^{2}\bar{\alpha}_{i}((e^{T}x)^{-1}x)=x^{T}(L_{i}+M_{i})x$, $i=1,...,n$, where $L_{i}$ and $M_{i}$ are non-negative and positive semi-definite, respectively. Therefore, we have

[TABLE]

where $L_{0}\geq 0$ and $M_{0}\succeq 0$. As nonnegative diagonal matrices are positive semi-definite and convexity of nonnegative and positive semi-definite matrices, with a little algebra, we get symmetric matrices $\bar{K}_{i}$ , $i=1,...,n$, which satisfy (18)-(20). As a result, $\bar{\ell}\leq p^{1}_{Q}$.

Now, we prove that $p^{1}_{Q}$ is less than or equal to the optimal value of (3). Similar to the former case, we assume that optimal solution is attained. So, there exist symmetric matrices $K_{i}$, $i=1,...,n$, which satisfy (18)-(20) for $B=Q-p^{1}_{Q}ee^{T}$. Let $M_{i}=Q-p^{1}_{Q}ee^{T}-K_{i}$, $i=1,...,n$. We have

[TABLE]

Hence,

[TABLE]

which completes the proof. ∎

The proof of above theorem reveals for bound (3), we can replace $\mathcal{Q}_{+}^{c}(\Delta)$ by the set of homogeneous convex quadratics. So, (3) is equivalent to the problem

[TABLE]

We conclude the section by noting that bound (3) dominates semidefinite relaxations obtained in the Laserre Hierarchy with the same order of variables for QCQPs. Strictly speaking, as bound (3) applies $\mathcal{Q}_{+}^{c}(\mathcal{X})$ instead of $\mathcal{Q}_{+}^{c}(\mathbb{R}^{n})$, it can generate tighter bounds in comparison with the Laserre Hierarchy with the same order of variables. Moreover, bound (3) and RLT-level 2 are not necessarily relevant.

4 Computational results

As mentioned above, two important factors which determine the efficiency of a given bound are the quality of the generated bound and its computational time. In the section, we compare quadratic bound $\eqref{L2}$ and cubic bound$\eqref{B7}$. The aim of this section is not to compare the numerical performance all above-mentioned bounds comprehensively. We merely present numerical performance a quadratic and a cubic bound on some instances. The reason why we chose these bounds is that they are formulated in the same line with different types multipliers.

In order to evaluate the efficiency of these bounds, we generated 26 random QCQPs in the form (QCQP) in $\mathbb{R}^{20}$ with five non-convex quadratic constraints and two equality constraints. To solve semi-definite programs, we employed MOSEK in Matlab 2018b environment [17]. We applied YALMIP to pass bounds $\eqref{L2}$ and $\eqref{B7}$ to MOSEK [13]. In addition, we employed BARON to obtain optimal values [20]. Moreover, the computations were run on a Windows PC with Intel Core i7 CPU, 3.4 GHz, and 24GB of RAM.

Figure 1 shows the generated bounds via $\eqref{L2}$ and $\eqref{B7}$. In the figure, red line denotes the difference between optimal value and generated bound by $\eqref{L2}$, absolute gap, and blue denotes absolute gap for bound $\eqref{B7}$. Bound (3) were exact for nine examples. However, on average, the computational time for bounds $\eqref{L2}$ and $\eqref{B7}$ were $0.1$ and $3.8$ seconds, respectively.