A survey of hidden convex optimization

TL;DR

This survey reviews methods to uncover hidden convex structures in nonconvex optimization problems, highlighting their importance and discussing open challenges in the field.

Contribution

It systematically summarizes three key approaches for revealing convexity in nonconvex problems and identifies ten open research questions.

Findings

Three main methods for hidden convexification are summarized.

Highlights the significance of convex structures in nonconvex problems.

Raises ten open problems for future research.

Abstract

Motivated by the fact that not all nonconvex optimization problems are difficult to solve, we survey in this paper three widely-used ways to reveal the hidden convex structure for different classes of nonconvex optimization problems. Finally, ten open problems are raised.

Equations

$$\Vert A{\Vert }_p=\underset{x\ne 0}{\sup }\frac{\Vert Ax{\Vert }_p}{\Vert x{\Vert }_p}$$

$$\mathrm{Co}:=\{A\in {\mathbb{R}}^{n\times n}:A=A^T,x^{T}Ax\ge 0,\forall x\ge 0,x\in {\mathbb{R}}^n\}.$$

$$(\mathrm{G})\underset{\theta }{\max }\varphi (\theta ):=\min \{a_1\theta ,a_2{\delta }^2(\theta )\},$$

$${\delta }^2(\theta )$$

$$\max$$

$$(\mathrm{P})\min \left\{f(x):g_i(x)\le b_i,i=1,\dots ,m,x\in S\right\},$$

$$w(y):=\underset{x\in S}{\min }\left\{f(x):g_i(x)\le y_i,i=1,\dots ,m,x\in S\right\}$$

$$({\mathrm{P}}_p)\min \left\{f^p(x):g_i^p(x)\le b_i^p,i=1,\dots ,m,x\in S\right\}.$$

$$({\mathrm{P}}_{p,\mu })\min \left\{f(x):(g_i(x)+{\mu }_i{)}^p\le (b_i+{\mu }_i{)}^p,i=1,\dots ,m,x\in S\right\}.$$

$$(\mathrm{MVE})$$

$$\underset{M,z}{\min }$$

$$(\mathrm{LMP})$$

$$\underset{x\in C}{\max }$$

$$(\mathrm{GP})$$

$$\min$$

$$s.t.$$

$$(\mathrm{LF})$$

$$s.t.$$

$$\min$$

$$(\mathrm{NLF})$$

$$s.t.$$

$$(\mathrm{CLF})$$

$$s.t.$$

$$(\mathrm{SLF})$$

$$(\mathrm{SOC}{\mathrm{P}}_{\mathrm{LP}})$$

$$R_A(x)=\frac{x^{T}Ax}{x^{T}x},x\ne 0.$$

$$\underset{x\ne 0}{\sup }{R}_A(x)=\underset{x^{T}x=1}{\max }{x}^{T}Ax.$$

$$\underset{x^{T}x=1}{\max }{x}^{T}Ax$$

$$(\mathrm{KI})\underset{x\ne 0}{\max }\frac{(x^{T}Ax)(x^T{A}^{-1}x)}{(x^{T}x{)}^2},$$

$$\underset{e^{T}z=1,z\ge 0}{\max }\left(\sum _{i=1}^n{\lambda }_i{z}_i\right)\left(\sum _{i=1}^n{\lambda }_i^{-1}{z}_i\right),$$

$$\underset{e^{T}z=1,z\ge 0}{\max }\underset{t\in [t_1,t_2]}{\min }t\left(\sum _{i=1}^n{\lambda }_i{z}_i\right)+t^{-1}\left(\sum _{i=1}^n{\lambda }_i^{-1}{z}_i\right)$$

Full text

1 Introduction

Nowadays, convex programming becomes very popular due to not only its various real applications but also the charming property that any local solution also remains global optimality.

However, it does not mean that convex programming problems are easy to solve. First, it may be difficult to identify convexity. Actually, deciding whether a quartic polynomials is globally convex is NP-hard Ah13 . Second, evaluating a convex function is also not always easy. For example, the induced matrix norm

[TABLE]

is a convex function. But evaluating $\|A\|_{p}$ is NP-hard if $p\neq 1,2,\infty$ H09 . Third, convex programming problems may be difficult to solve. It has been shown in Bu09 that the general mixed-binary quadratic optimization problems can be equivalently reformulated as convex programming over the copositive cone

[TABLE]

Notice that checking whether $A\in{\rm Co}$ is NP-Complete M87 .

Like a coin has two sides, there are quite a few nonconvex optimization problems could be globally and efficiently solved in polynomial time. The reason behind this observation is that most of them belong to the hidden convex optimization, i.e., they admit equivalent convex programming reformulations. In the past two decades, hidden convex optimization problems were studied case by case in literature. In this survey, we summarize three class of approaches to reveal the hidden convex structure, namely, nonlinear transformation, Lagrangian dual and the primal tight convex relaxation.

The remainder of this paper is organized as follows. Section 2 presents the most common used nonlinear transformations for different classes of problems. Section 3 shows how Lagrangian duality and its variations achieve zero gap. Section 4 summarizes some classes of nonconvex optimization problems admitting tight primal relaxations. As a concluding remark, ten open problems are raised in Section 5.

Throughout the paper, let $v(\cdot)$ be the optimal value of problem $(\cdot)$. $\mathbb{R}^{n}_{+}$ denotes the nonnegative orthant in $\mathbb{R}^{n}$. For a matrix $A$, denote by $A\succ(\succeq)0$ that $A$ is positive (semi)definite. The trace of $A$ is defined as the sum of its diagonal elements, i.e., ${\rm tr}(A)=\sum_{i=1}^{n}A_{ii}$. $\lambda_{\min}(A)$ and $\lambda_{\max}(A)$ denote the minimal and maximal eigenvalues of $A$, respectively. $I_{n}$ is the identity matrix of order $n$. $\|x\|=\sqrt{x^{T}x}$ stands for the $\ell_{2}$ norm of a vector $x$. ${\rm diag(x_{1},\ldots,x_{n})}$ returns a diagonal matrix with diagonal elements $x_{1},\ldots,x_{n}$. $e=(1,1,\ldots,1)^{T}\in\mathbb{R}^{n}$. Denote by ${\rm conv}\{\Omega\}$ the convex hull of $\Omega$. $|E|$ denotes the number of elements in the set $E$. For a real value $x$, [$x$] returns the largest integer less than or equal to $x$.

2 Nonlinear transformation

In this section, we survey some nonlinear transformation approaches widely used in convexifying different classes of nonconvex optimization problems.

2.1 A univariate example

Univariate examples are not always as simple as converting the concave function $\sqrt{x}$ to $y(\geq 0)$ by introducing the one-to-one mapping $x=y^{2}$ with $y\geq 0$.

The approach X11 for reducing the duality gap for box constrained nonconvex quadratic program requires solving the following nonconvex subproblem

[TABLE]

where $a_{1}$, $a_{2}$ are two positive scalars,

[TABLE]

$C$ is a linear manifold and $\omega_{i}\in\{-1,1\}$ is given. The objective function $\phi(\theta)$ (1) is not concave and thus (G) is a nonconvex minimization. Introducing $\sigma=\sqrt{1-\theta}$, one can reformulate $({\rm G})$ as the following convex program:

[TABLE]

2.2 $p$-th power approach

Consider the following constrained nonconvex optimization problems:

[TABLE]

where $f(x)$, $g_{i}(x)$ ($i=1,2,\ldots,m$) are strictly positive over $S$ and $S$ is closed, bounded, connected and of a full dimension. It is known L70 that if the perturbation function

[TABLE]

is locally convex in a closed and non-degenerate neighbourhood of $b$, then there is no duality gap between $(P)$ and its Lagrangian dual in the sense that the inner minimization is local and the outer maximization is in the neighbourhood of the optimal Lagrangian multiplier. In order to achieve such a zero duality gap, Li Li95 first introduced the $p$-th power transformation:

[TABLE]

Under some additional assumptions, Li Li95 showed that, for a sufficiently large $p$, the perturbation function of $({\rm{P}}_{p})$ is a convex function of $y^{p}$ for any $y$ in a neighbourhood of $b$.

For more applications of the $p$-th power formulation, we refer to L06 ; L07 and references therein. More generalizations of the $p$-th power convexification approach are further studied in L05 ; Wu07 .

Recently, a novel shifted $p$-th power reformulation is introduced in X16 :

[TABLE]

Surprisingly, under the same assumptions, there is a parametric vector $\mu\in\mathbb{R}^{m}$ such that $p=3$ is sufficient to guarantee the convexity of the perturbation function of $({\rm{P}}_{p,\mu})$ in terms of $(y+\mu)^{p}$ with $y$ lying in a neighbourhood of $b$. Besides, the assumption on the strict positiveness of $f(x)$ and $g_{i}(x)$ is redundant in this new reformulation. For more details, we refer to X16 .

2.3 Minimal-volumn ellipsoid cover

Consider the geometric problem of finding $n$-dimensional ellipsoid of minimal volume covering a set of $m$ given points $a_{i},~{}i=1,\ldots,m$:

[TABLE]

where $u_{n}$ is the volume of the unit ball in $\mathbb{R}^{n}$ and the objective is the $n$-dimensional volume.

By first introducing $M=Q^{\frac{1}{2}}$ (which is well defined as $Q\succ 0$) and $z=Mx$, and then taking a logarithmic transformation for the objective function, we can reformulate the nonconvex optimization problem (MVE) as the following convex program S04 :

[TABLE]

where the fact det$(Q^{-1})=($det$(M))^{-2}$ is used.

2.4 Multiplicative programming

Consider the linear multiplicative programming K95 :

[TABLE]

where $C$ is a convex set. (LMP) is NP-hard M96 to solve if replacing the maximization with minimization. The objective function of (LMP) is nonconcave but quasiconcave K95 . The hidden convexity of (LMP) is viewed by the equivalent convex programming problem in the sense that both sharing the same optimal solution:

[TABLE]

Moreover, we notice that (LMP) has the following new second-order cone programming representation:

[TABLE]

2.5 Geometric programming

Geometric program was first introduced in the book D67 . It is an optimization problem of the form

[TABLE]

where $g_{j}~{}(j=1,\ldots,p)$ are monomials, i.e. the form $c_{j}\prod_{i=1}^{n}x_{i}^{a_{ij}}$ with $c_{j}>0$, and $f_{i}~{}(i=0,\ldots,m)$ are posynomials, i.e., the sum of several monomials. We also assume that the constraints (9)-(9) implicitly imply that all the variables are positive.

Obviously, the general (GP) is a nonconvex optimization problem. One can verify that introducing logarithmic change of variables $y_{i}=\log(x_{i})$ and logarithmic transformations of $f_{i}(x)$ yields a convex programming reformulation of (GP):

[TABLE]

where $(e^{y})_{i}=e^{y_{i}}$.

2.6 Fractional programming

Consider the linear fractional program

[TABLE]

To guarantee that (LF) is well defined, we assume $\min_{x\in\Omega}d^{T}x+\beta>0$. It is trivial to verify that the objective function $f(x)$ (13) is quasi-convex and generally nonconvex. Introducing new variables $y$ and $t$ to replace $\frac{x}{d^{T}x+\beta}$ and $\frac{1}{d^{T}x+\beta}$, respectively, we can equivalently reformulate (LF) as the following linear programming:

[TABLE]

The above approach was initially suggested by Charnes and Cooper C62 . It was then extended by Schaible S74 to nonlinear convex fractional programs:

[TABLE]

where $g_{i}(x)$ ($i=0,1,\ldots,m$) are all convex functions, $h(x)$ is a concave function and $h(x)>0$ over the feasible region. Then, introducing $y=\frac{x}{h(x)}$ and $t=\frac{1}{h(x)}$ reduces the nonconvex optimization problem (NLF) to the following convex programming:

[TABLE]

Minimizing the sum of a linear and a linear fractional function over a polyhedral set is generally NP-hard M96 . Necessary and sufficient condition for the pseudoconcavity of the objective function is established in C08 . The pseudoconvex but nonconvex case is the following:

[TABLE]

Based on the Charnes-Cooper transformation, it was shown in FG2016 that (SLF) enjoys a second-order cone programming reformulation:

[TABLE]

2.7 Eigenvalue and trust-region subproblem

Define the Rayleigh quotient of a symmetric matrix $A\in\mathbb{R}^{n\times n}$ as:

[TABLE]

The Rayleigh-Ritz formula is well known for the maximal eigenvalue of $A$:

[TABLE]

Let $A=UDU^{T}$ be the eigenvalue decomposition of $A$, where $U$ is orthogonal and $D={\rm diag}(\lambda_{1},\ldots,\lambda_{n})$ with $\lambda_{1}\leq\ldots\leq\lambda_{n}$ being the $n$ eigenvalues of $A$. Then, we have

[TABLE]

where $y=U^{T}x$ is replaced in the first equality and it holds that $\|y\|=\|x\|$ as $U$ is orthogonal, the nonlinear transformation $z_{i}=y_{i}^{2}$, $i=1,\ldots,n$ are introduced in the last equality and we note that these mappings are no longer one-to-one. Problem on the right-hand side of the above chain equalities is a linear programming over a standard simplex, which serves as the hidden convex reformulation of (20).

Now we move to the well-known Kantorovich inequality, which is related to the following nonconvex optimization:

[TABLE]

where $A\succ 0$. Based on the same transformation as above, (KI) is reduced to

[TABLE]

which is further equivalent (in the sense that both optimal solutions are the same) to solving

[TABLE]

where $t_{1}=\sqrt{\lambda_{1}/\lambda_{n}}$, $t_{2}=t_{1}^{-1}$, and (21) follows from the classical von Neumann’s minimax theorem. Problem (22) is a univariate convex optimization problem and can be explicitly solved.

The inhomogeneous extension of (20) is the well-known trust-region subproblem (TRS) G81 :

[TABLE]

where $\delta$ is a positive scalar. (TRS) (23) plays a great role in the trust-region method Y15 and also has some other applications such as the constrained least squares G91 . In the case that $A\not\succeq 0$, (TRS) is a nonconvex optimization. Interestingly, it was shown in M94 that (TRS) has at most one local non-global minimizer.

As above, let $A=U{\rm diag}(\lambda_{1},\ldots,\lambda_{n})U^{T}$ be the eigenvalue decomposition of $A$ and $\widetilde{b}=U^{T}b$. By introducing $y=U^{T}x$, (TRS) is equivalent to

[TABLE]

For $i=1,\ldots,n$, by further introducing

[TABLE]

we can reduce (TRS) (24) to the following convex programming over a simplex:

[TABLE]

The above convexification approach is further extended by Ben-Tal and Teboulle B96 to the two-sided trust-region subproblem S95 . Similar application in convexifying the following regularized problem

[TABLE]

with $p>2$ can be found in H17 .

2.8 Quadratic matrix programming over orthogonal constraints

For vectors $u$ and $v$, define the minimal product as

[TABLE]

where $\pi$ is a permutation of $1,2,\ldots,n$. Notice that $\langle u,\ v\rangle_{-}$ is easy to solve by first sorting $u$ and $v$, respectively. $\langle u,\ v\rangle_{+}$ is similarly defined and computed:

[TABLE]

The quadratic assignment problem (QAP) is a classical combinatorial optimization problem L97 . The trace formulation reads as follows:

[TABLE]

where $A,B\in\mathbb{R}^{n\times n}$ correspond to flow matrix and distance matrix in a facility location application, respectively, ${\rm\Pi}_{n}$ is the set of all $n\times n$ permutation matrices. The orthogonal relaxation of (QAP) was first proposed in F87 :

[TABLE]

Let $\lambda$ and $\mu$ be the vectors composed by the eigenvalues of $A$ and $B$, respectively, i.e.,

[TABLE]

It is not difficult to show the following result.

Theorem 1** (F87 ; R92 )**

For any $X\in\mathcal{O}_{n}$, it holds that

[TABLE]

The hidden convex reformulation of (O) reads as follows:

[TABLE]

where $Y=U^{T}XV$ is introduced in (31) so that $Y$ remains orthogonal, and in (33), $z_{ij}=y_{ij}^{2}$ $(i,j=1,\ldots,n)$ are introduced. Notice that the problem (33) is a linear assignment problem. Therefore, there is an optimal solution, denoted by $Z^{*}$, lying at one of the vertices. That is, for all $i,j$, we have $z_{ij}^{*}\in\{0,1\}$. It follows that $y_{ij}^{*2}\in\{0,1\}$ and hence $y_{ij}^{*}\in\{0,1\}$, which further implies that $Y^{*}\in{\rm\Pi}_{n}\subseteq\mathcal{O}_{n}$. Therefore, the inequality (32) holds as an equality.

This convexification approach could be further extended to the trust-region type relaxation A99 by replacing (30) with

[TABLE]

and the enhanced version X11a with (30) being replaced by

[TABLE]

where $m$ is an integer between [math] and $n$.

3 Lagrangian dual and its variation

In this section, we first show that strong Lagrangian duality could hold for a few nonconvex optimization problems or their special cases. Sometimes, in order to achieve strong duality, the approach such as adding redundant constraints or making suitable transformation should be introduced in advance.

3.1 Strong Lagrangian duality

Let us begin with the generalized trust-region subproblem with interval bounds P14 :

[TABLE]

where $f_{i}(x)=x^{T}A_{i}x+2a_{i}^{T}x+c_{i}$ for $i=1,2$, $A_{1},A_{2}\in\mathbb{R}^{n\times n}$ are symmetric matrices, $a_{1},a_{2}\in\mathbb{R}^{n}$ and $c_{1},c_{2}\in\mathbb{R}$. We make a further assumption:

Assumption 1

$A_{2}\neq 0$, $-\infty<\alpha\leq\beta<+\infty$ and there are $y,z\in\mathbb{R}^{n}$ such that either $\alpha<f_{2}(y)<\beta$ or $f_{2}(y)<\alpha=\beta<f_{2}(z)$ (i.e., the primal Slater condition holds).

A real application is the squared least squares model for the global positioning system (GPS) location B12 :

[TABLE]

which can be reformulated as a special case of (QP):

[TABLE]

A key technique to establish the strong duality is the following S-lemma with interval bounds W15 , which generalizes the classical S-lemma PT07 and the S-lemma with equality XSR16 .

Theorem 2** (S-lemma with interval bounds W15 )**

Under Assumption 1, the system $f_{1}(x)<0,~{}\alpha\leq f_{2}(x)\leq\beta$ is unsolvable if and only if there is a $\mu\in\mathbb{R}$ such that $f_{1}(x)+\mu_{-}(f_{2}(x)-\beta)+\mu_{+}(\alpha-f_{2}(x))\geq 0,~{}\forall x\in\mathbb{R}^{n}$, where $\mu_{+}=\max\{\mu,0\}$ and $\mu_{-}=\min\{\mu,0\}$.

Then we show that the strong Lagrangian duality holds for (QP), which provides a hidden convexity of (QP) from the dual side.

[TABLE]

where (34) follows from Theorem 2 and the last equality is based on the trivial observation

[TABLE]

When $A_{1}$ and $A_{2}$ are diagonal or simultaneous diagonalizable, (QP) admits a second-order conic reformulation BT14 , which seems to be easier to solve than the SDP. Extensions to the general case can be found in J17 .

To study quadratic program with two nonindependent quadratic constraints, we need the following general S-procedure due to Polyak Poly .

Theorem 3** (Theorem 4.1, Poly )**

Let $n\geq 3$ and $A_{0},A_{1},A_{2}$ be $n\times n$ real symmetric matrices. Suppose there are scalars $\mu_{1},\mu_{2}$ and $\widetilde{x}\in\mathbb{R}^{n}$ such that

[TABLE]

Then, the system

[TABLE]

has no solution if and only if there exist $\lambda_{1}\geq 0$, $\lambda_{2}\geq 0$ such that

[TABLE]

Consider the homogeneous nonconvex quadratic constrained quadratic programming:

[TABLE]

Under the assumption (39)-(40), strong duality of (QP2) is verified as follows:

[TABLE]

Then we get a semidefinite programming reformulation of (QP2).

Strong duality of (QP2) can be extended to the problem with two equality constraints under the Slater’s assumption for equalities. As a small application, consider the binary quadratic programming problem

[TABLE]

which is NP-hard as it contains the classical Max-Cut problem as a special case. The Lagrangian dual is given by

[TABLE]

An optimal parametric Lagrangian dual approach is proposed in X15 , which implies that

[TABLE]

Here, one can observe that (BQP) with $n\leq 2$ is just a special case of (QP2) with equality constraints and hence the strong duality holds true.

Note that (QP2) is homogeneous. However, for the inhomogeneous case, there could be a positive duality gap. A well-known example is the Celis-Dennis-Tapia (CDT) subproblem (see Section 5). But if the variables are in the complex field, strong duality holds again BE06 .

3.2 Adding redundant constraints

Consider the problem (O) (30)-(30), the orthogonal relaxation of (QAP). Above we have shown that (O) enjoys hidden convexity. However, as pointed in Z98 , the standard Lagrangian duality gap could be positive. Another counter example can be found in W02 . Interestingly, the duality gap is closed by adding a redundant constraint A00 . More precisely, (O) is equivalent to

[TABLE]

The Lagrangian dual problem of (O2) is

[TABLE]

It was shown in A00 ; W02 that $v({\rm O2})=v({\rm DO2})$ and strong duality holds again.

Strong duality is achieved for the trust-region-type relaxation with additional constraints X11a :

[TABLE]

where $m\geq 0$ is an integer. The special case $m=0$ was first studied in A99 . Without the redundant constraint (3.2), there could be a positive duality gap, see X11a . For other applications of the similar approach, we refer to D11 .

The other example is the univariate polynomial optimization

[TABLE]

where $a_{2k}>0$ so that v(PO)$>-\infty$. Let $x_{i}=x^{i}$. (PO) can be reformulated as the following quadratic program with many redundant quadratic constraints:

[TABLE]

Theorem 4** (S98 )**

The Lagrangian dual value of the quadratic constrained quadratic program (45)-(47) is equal to v$({\rm PO})$.

The approach was further extended to the multidimensional case with sum-of-square structure S98 .

3.3 Scaled Lagrangian duality

3.3.1 Quadratic programming

The standard quadratic programming is a nonconvex quadratic program over the standard simplex:

[TABLE]

where $Q\in\mathbb{R}^{n\times n}$ and $Q\not\succeq 0$. Any general quadratic function $x^{T}Qx+2b^{T}x$ over $\Delta$ can be homogenized by rewriting $b^{T}x=x^{T}(eb^{T})x$ and $x^{T}b=x^{T}(be^{T})x$ so that $x^{T}Qx+2b^{T}x=x^{T}(Q+eb^{T}+be^{T})x$. Problem (QPS) is NP-hard since the maximum stability number can be reformulated in (QPS) M65 .

We can equivalently reformulate (QPS) as (see X13c )

[TABLE]

The Lagrangian dual problem of ${\rm(QPS_{1})}$ reads

[TABLE]

The strong duality can be verified that A10

[TABLE]

It should be noted that there is a positive gap between (QPS) and its Lagrangian dual problem even when $n=5$.

3.3.2 Fractional programming

Reconsider the nonlinear convex fractional programming (NLF) (15)-(16). As shown in S76 , the standard Lagrangian dual is not tight for relaxing (NLF). A natural way to define a dual problem with no duality gap is to write the Lagrangian dual of the equivalent convex reformulation (CLF) (17)-(19), which reads as follows S74 ; S76 :

[TABLE]

In other words, (NLF) with the following scaling

[TABLE]

enjoys strong duality.

The assumption that $g_{i}(x)$ $(i=1,\ldots,m)$ are convex and $h(x)>0$ is concave is not necessary in establishing strong duality of the above problem (NLF). Consider the quadratic fractional programming over an interval quadratic constraint:

[TABLE]

where $f_{i}(x)$ ($i=0,1,2$) are quadratic functions. Suppose Assumption 1 holds. To guarantee a well-defined (QPF), we further assume $\inf_{x\in\Omega}f_{1}(x)>0$. Special cases of the problem (QPF) were studied in B09 ; N16 ; X15 . In (QPF), $f_{0}(x)$ is nonconvex and $f_{1}(x)$ is nonconcave.

It was proved in Y17 that the equivalent reformulated optimization problem

[TABLE]

achieves zero Lagrangian duality gap. However, without this scaling, (QPF) could have a positive Lagrangian duality gap. Consider a special case of (QPF), the identical regularized total least squares problem (TLS) B06 :

[TABLE]

where $A\in\mathbb{R}^{m\times n}$ ($m\geq n$) and $0\leq\alpha\leq\beta<+\infty$ is assumed. The necessary and sufficient condition for the strong duality was established in Y17 .

Theorem 5

The Lagrangian duality gap for $({\rm ITLS})$ is positive if and only if

[TABLE]

3.3.3 Orthogonal constrained linear optimization

Replacing the objective function in (O) (30)-(30) with a linear one yields the following problem:

[TABLE]

It is easy to verify that the optimal value of (LO) is equal to the sum of all singular values of $C$. However, there are examples such that the gap between (LO) and its Lagrangian dual is positive W02 . To close the duality gap, (LO) is equivalently rewritten as

[TABLE]

where W=\left(\begin{array}[]{cc}X&Y\\ V&Z\end{array}\right). Then, we obtain (O2).

4 Tight primal relaxation

Primal relaxation could provide an alternative way to reveal the hidden convexity of the original nonconvex optimization problem.

4.1 Totally unimodular

Consider the integer programming problem

[TABLE]

where $b\in Z^{n}$ and $Z^{n}$ is the field of integer vectors. The matrix $M$ is called totally unimodular if each determinant of any square submatrix is $0,1$ or $-1$. It follows that the linear programming relaxation

[TABLE]

has an integral optimal solution. Instances of this class include the min-cut problem, the linear assignment problem and so on.

4.2 Orthogonal constrained problems

We first consider a slightly generalized version of the linear problem (LO) (50)-(51):

[TABLE]

where $k\in\{1,\ldots,n\}$ and $C\in\mathbb{R}^{n\times k}$. (LO) (50)-(51) exactly corresponds to the special case $k=n$. (GLO) is trivially reformulated as the following convex programming:

[TABLE]

which can be reduced to a semidefinite program as one can verify that

[TABLE]

Consider the relaxation problem (O) (30)-(30). It is trivial to see that (O) can be equivalently rewritten as a linear problem:

[TABLE]

The first computable representation of the convex set $\mathcal{O}(B)$ was established in X13 as follows:

Theorem 6** (X13 )**

[TABLE]

Then, we obtain a semifinite programming reformulation of (O).

4.3 Trust-region subproblem and extensions

The trust-region subproblem (TRS) (23) admits the following primal convex quadratic optimization reformulation F96

[TABLE]

When extended to the two-sided trust-region subproblem

[TABLE]

the following equivalent convex reformulation was established in W17 :

[TABLE]

Applying Nesterov s accelerated gradient descent algorithm for solving (54)-(55) yields a linear-time algorithm N17 ; W17 , whose worst-case complexity is less than the previously existing algorithm H16 . Recently, the extended (TRS) where (55) is replaced by a general quadratic constraint is convexified in Y18 .

Besides (TRS), some other optimization problems over the unit-ball also have hidden convexity. For example, the ball-constrained weighted maximin dispersion problem Ha13

[TABLE]

where $x^{1},\ldots,x^{m}\in\mathbb{R}^{n}$ are given $m$ points and $\omega_{i}>0$ for $i=1,\ldots,m$. Problem (DP) is NP-hard W16 . However, when $m\leq n$, (DP) is equivalent to the following second-order cone programming problem W16 :

[TABLE]

The other example comes from the identical Tikhonov regularized total least squares BB06 ; B06

[TABLE]

According to the S-lemma with equality XSR16 , a special case of Theorem 2 with the setting $\alpha=\beta$, we have

[TABLE]

The above SDP reformulation can be equivalently reduced to finding the unique zero point of a smooth, strictly decreasing and convex univariate function in terms of $\mu$. For more details, see Y17b .

Moreover, (TRS) (23) itself has the following equivalent semidefinite programming reformulation R97 :

[TABLE]

The basic idea behind this equivalence is Pataki’s theorem P98 .

Theorem 7** (P98 )**

Suppose the SDP problem

[TABLE]

has a solution and assume

[TABLE]

Then, for (SDP), there is an optimal solution $X^{*}$ such that rank($X^{*}$)$\leq r$.

Theorem 7 implies that (QP2) (41)-(42) admits a tight primal SDP relaxation.

(TRS) was further generalized by adding several linear cuts:

[TABLE]

When $m=1$, the hidden convex reformulation of $({\rm T_{1}})$ was given in S03 ; Y03 :

[TABLE]

where the additional second-order constraint (4.3) was obtained by linearizing the valid constraint:

[TABLE]

Further extension such as letting $\delta$ be an additional variable is studied in G14 .

Suppose $m=2$ and the two linear cuts are parallel, without loss of generality, we assume the constraints (62) are

[TABLE]

Then, $({\rm T_{2}})$ is equivalent to the following SOC-SDP problem B13 :

[TABLE]

where (4.3) corresponds to linearizing the valid constraint:

[TABLE]

Generally, when all the linear cuts (62) are non-intersecting in the ball (62), $({\rm T_{m}})$ has the following SOC-SDP reformulation B15 :

[TABLE]

Besides, for the intersection of an ellipsoid and a split disjunction, the convex hull is recently shown to be second-order-cone representable B17 .

4.4 Quadratic matrix programming

Consider a class of quadratic matrix programming Be06 :

[TABLE]

where $A_{i}=A_{i}^{T}\in\mathbb{R}^{n\times n}$, $B_{i}\in\mathbb{R}^{n\times s}$ ($i\in\{0\}\cup I\cup E$) and $V\in R^{s\times r}$ with $s\leq r$. Applications of this model include robust least squares and the sphere-packing problem.

For $i\in\{0\}\cup I\cup E$, define

[TABLE]

Relaxing \left(\begin{array}[]{c}X\\ V\end{array}\right)(X^{T}~{}~{}V^{T}) to $Z$ yields the following SDP:

[TABLE]

As an application of Theorem 7, it was shown in BD12 that the above primal SDP relaxation is tight if its optimal value is attainable and either $n+s\leq r$ or $|I|+|E|\leq\frac{(r+2)(r+1)}{2}-\frac{s(s+1)}{2}-1$.

For more nonconvex instances with tight SDP relaxation, we refer to L16 ; L13 ; L97 ; Y18 and references therein.

5 Open problems

We conjecture that any polynomially solved optimization problem has an equivalent hidden convex reformulation. Nevertheless, we are expected to find the hidden convex reformulations of the following ten special nonconvex optimization problems in the near future.

Open Problem 1. The Celis-Dennis-Tapia (CDT) subproblem

[TABLE]

where $Q_{1}\succ 0$. Recently, the polynomial-time solvability of (CDT) has been proved in B16 ; CL2016 ; S16 . Hidden convex reformulation even for the diagonal (CDT) (i.e., $Q_{0},Q_{1}$ and $Q_{2}$ are all diagonal matrices) remains unknown.

Open Problem 2. Finding the hidden convex reformulation of the extended trust-region subproblem with any fixed number of linear cuts

[TABLE]

which is polynomially solved when $k$ is fixed B14 ; H13 .

Open Problem 3. The regularized version of the trust-region subproblem with $k$ linear cuts reads as follows:

[TABLE]

where $p>2$ is a parameter. When $k=0$, there is at most one local non-global minimizer H17 . Hidden convexity with any fixed $k\geq 1$ is unknown.

Open Problem 4. The unbalanced orthogonal Procrustes problem

[TABLE]

Notice that when $k=1$, it reduces to trust-region subproblem. And when $k=n$, it reduces to the balanced case with an explicit solution S66 and admits a quadratic matrix programming Be06 . Hidden convex reformulation for the special cases $k=2$ and $k=n-1$ are expected.

Open Problem 5. Finding the hidden convex reformulation of the Tikhonov regularized total least squares problem BB06

[TABLE]

which is polynomially solved as it can be reformulated as a special case of the generalized (CDT). The special case $L=I$ is settled in Y17b .

Open Problem 6. Finding the hidden convex reformulation of the sum of a generalized Rayleigh quotient and a quadratic form on the unit sphere

[TABLE]

where $B\succ 0$. This problem is a generalization of the Rayleigh quotient optimization problem. It was raised in Z14 with applications in the downlink of a multi-user MIMO system and the sparse Fisher discriminant analysis in pattern recognition.

Open Problem 7. Let $A,B$ be $n\times n$ positive definite matrices and $0\neq b\in\mathbb{R}^{n}$. The special unconstrained quartic minimization

[TABLE]

has special application in the Legendre-Fenchel conjugate of the product of two positive definite quadratic functions H07 . Assuming the objective function being convex, it has been solved in Z10 . It is further shown in X13a that the convexity assumption could be removed. However, hidden convexity of this problem remains unknown.

Open Problem 8. Let $A_{k}\in\mathbb{R}^{n\times n}$ be symmetric for $k=1,2,\ldots,m$. The ball-constrained quartic minimization

[TABLE]

is generally NP-hard N03 . Hidden convexity even for the special case $k=2$ remains unknown.

Open Problem 9. Let $A_{k}\in\mathbb{R}^{n\times n}$ be symmetric for $k=1,2,\ldots,m$. The optimal value of

[TABLE]

plays a great role in the local convex analysis for quadratic transformations X14 . When $m=1$, the optimization problem reduces to the maximal eigenvalue of $A_{1}^{T}A_{1}$. Hidden convexity for fixed $m$ (say $m=2$) is unknown.

Open Problem 10. Let $\Omega=\{x\in\mathbb{R}^{n}:\|x-a_{i}\|^{2}\leq r_{i}^{2},~{}i=1,\ldots,p\}$ be the intersection of $p$ balls. Finding the Chebyshev center of $\Omega$ is modeled as

[TABLE]

Geometrically, $({\rm CC_{B}})$ is to find the smallest ball enclosing $\Omega$. When $p\leq n$, $({\rm CC_{B}})$ admits a standard quadratic programming representation Be07 ; Be09 . Hidden convexity for fixed $n$ or $p=n+1$ remains unknown.