Polyhedral approximations of the semidefinite cone and their application

TL;DR

This paper introduces a new sparse polyhedral approximation of the semidefinite cone using expanded SD bases, improving efficiency in solving semidefinite relaxations of combinatorial problems.

Contribution

It proposes an expanded SD basis for polyhedral approximation that maintains sparsity and enhances computational efficiency in semidefinite programming.

Findings

Approximation contains diagonally dominant matrices

Approximation is contained in scaled diagonally dominant matrices

Methods outperform existing approaches in efficiency

Abstract

We develop techniques to construct a series of sparse polyhedral approximations of the semidefinite cone. Motivated by the semidefinite (SD) bases proposed by Tanaka and Yoshise (2018), we propose a simple expansion of SD bases so as to keep the sparsity of the matrices composing it. We prove that the polyhedral approximation using our expanded SD bases contains the set of all diagonally dominant matrices and is contained in the set of all scaled diagonally dominant matrices. We also prove that the set of all scaled diagonally dominant matrices can be expressed using an infinite number of expanded SD bases. We use our approximations as the initial approximation in cutting plane methods for solving a semidefinite relaxation of the maximum stable set problem. It is found that the proposed methods with expanded SD bases are significantly more efficient than methods using other existing approximations or solving semidefinite relaxation problems directly.

Tables

Table 1. Table 1: Specifications of the experimental methods

Method	${𝒫}^n$	Number of cuts added at each iteration		Solver
		LP cut	SOCP cut
CPDD	$𝒟{𝒟}_n^{\ast }$	2	0	Gurobi
CPSDB	$𝒮𝒟{ℬ}_n^{\ast }$	2	0	Gurobi
CPSDD	$𝒮𝒟{𝒟}_n^{\ast }$	2	0	Mosek
SDSOS	$𝒮𝒟{𝒟}_n^{\ast }$	2	1	Mosek

Table 2. Table 2: Upper bounds obtained by SDP and SOCP methods on E ​ R ​ ( n , p ) 𝐸 𝑅 𝑛 𝑝 ER(n,p) graphs

n	p	CPSDD0/SDSOS0		CPSDD		SDSOS		SDP
		Value	Time (s)	(5 min)	(10 min)	(5 min)	(10 min)	Value	Time (s)
150	0.3	105.70	0.95	38.91	37.02	40.97	37.38	20.44	105.46
150	0.8	31.78	1.00	10.07	9.66	9.70	9.31	6.00	110.63
200	0.3	140.47	3.14	70.48	55.52	75.46	61.31	23.73	549.63
200	0.8	40.92	3.14	12.10	11.29	12.17	11.38	6.45	497.55
250	0.3	176.25	6.60	115.41	93.81	119.67	99.99	26.78	1562.52
250	0.8	51.87	6.79	17.36	15.30	17.43	15.39	7.18	1553.63
300	0.3	210.32	13.05	160.42	138.60	162.77	143.12	(29.13)	(32300.60)
300	0.8	60.97	13.31	21.71	17.77	22.66	18.50	(7.65)	(20586.02)

Table 3. Table 3: Upper bounds obtained by LP methods on the same E ​ R ​ ( n , p ) 𝐸 𝑅 𝑛 𝑝 ER(n,p) graphs

n	p	CPDD0		CPDD		CPSDB0		CPSDB
		Value	Time (s)	(5 min)	(10 min)	Value	Time (s)	(5 min)	(10 min)
150	0.3	117	0.06	76.76	67.51	107.29	0.24	36.80	35.12
150	0.8	46	0.05	13.70	12.71	32.76	0.28	9.51	9.06
200	0.3	157	0.1	113.28	104.07	142.25	0.52	55.07	48.18
200	0.8	54	0.11	17.39	16.07	42.14	0.57	11.58	11.00
250	0.3	194	0.17	154.75	146.20	178.30	0.84	91.88	73.24
250	0.8	68	0.17	28.02	22.26	53.22	1.00	14.76	13.57
300	0.3	230	0.26	183.89	174.02	212.97	1.29	133.83	110.95
300	0.8	78	0.24	47.87	32.28	62.47	1.36	18.11	16.05

Equations

$$\min$$

$$s.t.$$

$$X\in {\mathcal{S}}_{+}^n,$$

$$\mathcal{F}\mathcal{W}(k):=\{X\in {\mathbb{S}}^n\mid \text{X has a factor width of at most }k\}.$$

$${\mathcal{DD}}_n$$

$${\mathcal{S}\mathcal{DD}}_n$$

$${\mathcal{B}}_{+}:=\{(e_i+e_j)(e_i+e_j{)}^T\mid 1\le i\le j\le n\}$$

$${\mathcal{B}}_{-}:=\{(e_i+e_i)(e_i+e_i{)}^T\mid 1\le i\le n\}\cup \{(e_i-e_j)(e_i-e_j{)}^T\mid 1\le i<j\le n\}$$

$$B_{i,j:}^{+}=(e_i+e_j)(e_i+e_j{)}^T,B_{i,j}^{-}:=(e_i-e_j)(e_i-e_j{)}^T.$$

$${\mathcal{S}}_{\mathrm{in}}:=\mathrm{cone}({\mathcal{B}}_{+}\cup {\mathcal{B}}_{-}),{\mathcal{S}}_{\mathrm{out}}:=({\mathcal{S}}_{\mathrm{in}}{)}^{\ast }.$$

$$P{\mathcal{B}}_{+}{P}^T:=\{P{B}_{i,j}^{+}{P}^T\mid B_{i,j}^{+}\in {\mathcal{B}}_{+}\}\text{mbox}andP{\mathcal{B}}_{-}{P}^T:=\{P{B}_{i,j}^{-}{P}^T\mid B_{i,j}^{-}\in {\mathcal{B}}_{-}\}$$

$$\min ⟨C,X⟩s.t.⟨A,X⟩=b,⟨Y,X⟩\ge 0(Y\in {\mathcal{B}}_{+}),$$

$$\min ⟨PC{P}^T,\bar{X}⟩s.t.⟨PA{P}^T,\bar{X}⟩=b,⟨Y,\bar{X}⟩\ge 0(Y\in P{\mathcal{B}}_{+}{P}^T).$$

$${\mathcal{S}}_{+}^n=\mathrm{cone}\left(\bigcup _{P\in {\mathcal{O}}^n}\{P^{T}XP\mid X\in {\mathcal{B}}_{+}\}\right)=\mathrm{cone}\left(\bigcup _{P\in {\mathcal{O}}^n}\{P^{T}XP\mid X\in {\mathcal{B}}_{-}\}\right),$$

$$\mathrm{cone}({\mathcal{B}}_{+}\cup {\mathcal{B}}_{-})={\mathcal{DD}}_n.$$

$${\bar{B}}_{i,j}(\alpha )$$

$$\bar{\mathcal{B}}(\alpha )$$

$${\bar{B}}_{i,j}(\alpha ):=$$

$$=$$

$${\bar{B}}_{i,i}(\alpha ):=$$

$$=$$

$$\sum _{1\le i\le j\le n}{\gamma }_{i,j}{\bar{B}}_{i,j}(\alpha )=O.$$

$$O=$$

$$=$$

$$+\sum _{j=2}^n\frac{\alpha (\alpha -1)}{4}(\sum _{i=1}^{j-1}{\gamma }_{i,j})B_{j,j}^{+}$$

$$=$$

$$+\sum _{i=2}^{n-1}\left[\frac{(1+\alpha {)}^2}{4}{\gamma }_{i,i}+\frac{1-\alpha }{4}(\sum _{j=i+1}^n{\gamma }_{i,j})+\frac{\alpha (\alpha -1)}{4}(\sum _{j=1}^{i-1}{\gamma }_{j,i})\right]B_{i,i}^{+}$$

$$+\left[\frac{{\gamma }_{n,n}(1+\alpha {)}^2}{4}+\frac{\alpha (\alpha -1)}{4}(\sum _{j=1}^{n-1}{\gamma }_{j,n})\right]B_{n,n}^{+}$$

$$+\sum _{1\le i<j\le n}\alpha {\gamma }_{i,j}{B}_{i,j}^{+}.$$

$$0=\frac{{\gamma }_{1,1}(1+\alpha {)}^2}{4}+\frac{1-\alpha }{4}(\sum _{j=2}^n{\gamma }_{1,j}),$$

$$0=\frac{(1+\alpha {)}^2}{4}{\gamma }_{i,i}+\frac{1-\alpha }{4}(\sum _{j=i+1}^n{\gamma }_{i,j})+\frac{\alpha (\alpha -1)}{4}(\sum _{j=1}^{i-1}{\gamma }_{j,i})(2\le i\le n-1),$$

$$0=\frac{{\gamma }_{n,n}(1+\alpha {)}^2}{4}+\frac{\alpha (\alpha -1)}{4}(\sum _{j=1}^{n-1}{\gamma }_{j,n}),$$

$$0=\alpha {\gamma }_{i,j}(1\le i<j\le n).$$

$${\gamma }_{i,j}=0(1\le i<j\le n).$$

$${\gamma }_{i,i}=0(i=1,2,\dots ,n).$$

$$(e_i+{\alpha }_2{e}_j)(e_i+{\alpha }_2{e}_j{)}^T\notin \mathrm{cone}(\bar{\mathcal{B}}({\alpha }_1)).$$

$${\bar{B}}_{i,j}^1:=(e_i+{\alpha }_1{e}_j)(e_i+{\alpha }_1{e}_j{)}^T,{\bar{B}}_{i,j}^2:=(e_i+{\alpha }_2{e}_j)(e_i+{\alpha }_2{e}_j{)}^T.$$

Full text

Key words: Semidefinite optimization problems; Conic optimization problems; Polyhedral approximation; Semidefinite bases; Expanded semidefinite bases.

AMS subject classifications: 90C05, 90C22, 90C25

1 Introduction

A semidefinite optimization problem (SDP) is an optimization problem in variables in the space of symmetric matrices with a linear objective function and linear constraints over the semidefinite cone. We denote the space of symmetric matrices as ${\mathbb{S}}^{n}:=\{X\in\mathbb{R}^{n\times n}\mid X_{i,j}=X_{j,i}\ (1\leq i<j\leq n)\}$ and the semidefinite cone as ${\cal S}^{n}_{+}:=\{X\in\mathbb{S}^{n}\mid d^{T}Xd\geq 0\ \mbox{for any}\ d\in\mathbb{R}^{n}\}$. Accordingly, we can readily define an SDP in the standard form, as

[TABLE]

where $C\in\mathbb{S}^{n}$, $A_{j}\in\mathbb{S}^{n}$, $b_{j}\in\mathbb{R}$ ($j=1,2,\ldots,m$), and $\langle A,B\rangle:={\rm Trace}(A^{T}B)=\sum_{i,j=1}^{n}A_{i,j}B_{i,j}$ is the inner product over $\mathbb{S}^{n}$.

SDPs are powerful tools that provide convex relaxations for combinatorial and nonconvex optimizations, such as the max-cut problem (e.g., [19], [12]) and the k-equipartition problem (e.g., [46], [23]). Some of these relaxations can even attain the optimum, as shown in [31] and [24]. Interested readers may find details about SDPs and their relaxations in [46], [42] and [32].

A cone ${\cal K}\subset\mathbb{S}^{n}$ is called proper if it has a non-empty interior and is closed, pointed (i.e., ${\cal K}\cap-{\cal K}=\{O\}$), and convex. It is known that the SDP cone is a proper cone [9]. By replacing the semidefinite constraint $X\in{\cal S}^{n}_{+}$ with a general conic constraint $X\in{\cal K}$ in (1) (say, a proper cone ${\cal K}\subset\mathbb{S}^{n}$), one can obtain a general class of problems, namely, conic optimization problems. The class of conic optimization problems has been an active field of study because it contains many popular classes of problems, including linear optimization problems (LPs), second-order cone programs (SOCPs), SDPs, and copositive programs. Copositive programs have been shown capable of providing tight lower bounds for combinatorial and quadratic optimization problems, as described in the survey paper by Dür [17] and the recent work of Arima et al. [3], [25], [4], etc. It has been shown that a copositive relaxation sometimes gives a highly accurate approximate solution for some combinatorial problems under certain conditions [5], [11]. However, the copositive program and its dual problem are both NP-hard (see, e.g., [16] and [36]).

SDPs are also attractive because they can be solved in polynomial time to any desired precision. There are state-of-the-art solvers, such as SDPA [47], SeDuMi [40], SDPT3 [43], and Mosek [35], but their computations become difficult when the size of the SDP becomes large. To overcome this deficiency, for example, one may use preprocessing to reduce the size of the SDPs, which leads to facial reduction methods [37], [38] and [44]. As another idea, one may generate relaxations of SDPs and solve them as easily handled optimization problems, e.g., LPs and SOCPs, which leads to cutting plane methods. We will focus on these latter methods.

The cutting plane method solves an SDP by transforming it into an optimization problem (e.g., an LP or an SOCP), adding cutting planes at each iteration to cut the current approximate solution out of the feasible region in the next iterations and to get close to the optimal value. The cutting plane method was first used on the traveling-salesman problem, by Dantzig, Fulkerson, and Johnson [13], [14] in 1954. It was used in 1958 by Gomory [20] to solve integer linear programming problems. As SDPs became popular, it came to be used on them as well; see, for instance, Krishnan and Mitchell [28], [30] and [29], and Konno et al. [27]. Kobayashi and Takano [26] applied it to a class of mixed-integer SDPs. In [1], Ahmadi, Dash, and Hall applied it to nonconvex polynomial optimization problems and copositive programs.

In the above-mentioned cutting plane methods for SDPs, the semidefinite constraint $X\in{\cal S}^{n}_{+}$ in (1) is first relaxed to $X\in{\cal K}_{\rm out}$, where ${\cal S}^{n}_{+}\subseteq{\cal K}_{\rm out}\subseteq{\mathbb{S}}^{n}$, and an initial relaxation of the SDP is obtained. If ${\cal K}_{\rm out}$ is polyhedral, the initial relaxation may give an LP; if ${\cal K}_{\rm out}$ is given by second-order constraints, the initial relaxation becomes an SOCP. To improve the performance of these cutting plane methods, we consider generating initial relaxations for SDPs that are both tight and computationally efficient and focus on approximations of ${\cal S}^{n}_{+}$.

Many approximations of ${\cal S}^{n}_{+}$ have been proposed on the basis of its well-known properties. Kobayashi and Takano [26] used the fact that the diagonal elements of semidefinite matrices are nonnegative. Konno et al. [27] imposed an assumption that all diagonal elements of the variable $X$ in the SDPs appearing in their iterative algorithm are bounded by a constant. The sets of diagonally dominant matrices and scaled diagonally dominant matrices are known to be cones contained in ${\cal S}^{n}_{+}$, (see, e.g., [22] and [1] for details). The inclusive relation among them has been studied in, e.g., [7] and [8]. Ahmadi et al. [1] and [2] used these sets as initial approximations of their cutting plane method. Boman et al. [10] defined the factor width of a semidefinite matrix, and Permenter and Parrilo used it to generate approximations of ${\cal S}^{n}_{+}$, which they applied to facial reduction methods in [37].

Tanaka and Yoshise defined various bases of $\mathbb{S}^{n}$, wherein each basis consists of $\frac{n(n+1)}{2}$ semidefinite matrices, called semidefinite (SD) bases, and used them to devise approximations of ${\cal S}^{n}_{+}$ [41]. They showed that the conical hull of SD bases and its dual cone give inner and outer polyhedral approximations of ${\cal S}^{n}_{+}$, respectively. On the basis of the SD bases, they also developed techniques to determine whether a given matrix is in the semidefinite plus nonnegative cone ${\cal S}^{n}_{+}+{\cal N}^{n}$, which is the Minkowski sum of ${\cal S}^{n}_{+}$ and the nonnegative matrices cone ${\cal N}^{n}$. In this paper, we focus on the fact that SD bases are sometimes sparse, i.e., the number of nonzero elements in a matrix is relatively small, and hence, it is not so computationally expensive to solve polyhedrally approximated problems in such SD bases. We call such an approximation, a sparse polyhedral approximation, and propose efficient sparse approximations of ${\cal S}^{n}_{+}$.

The goal of this paper is to construct tight and sparse polyhedral approximations of ${\cal S}^{n}_{+}$ by using SD bases in order to solve hard conic optimization problems, e.g., doubly nonnegative (DNN, or ${\cal S}^{n}_{+}\cap\mathcal{N}^{n}$) and semidefinite plus nonnegative ($\mathcal{S}^{n}_{+}+\mathcal{N}^{n}$) optimization problems. The contributions of this paper are summarized as follows.

• •

This paper gives the relation between the conical hull of sparse SD bases and the set of diagonally dominant matrices. We propose a simple expansion of SD bases without losing the sparsity of the matrices and prove that one can generate a sparse polyhedral approximation of ${\cal S}^{n}_{+}$ that contains the set of diagonally dominant matrices and is contained in the set of scaled diagonally dominant matrices.

• •

The expanded SD bases are used by cutting plane methods for a semidefinite relaxation of the maximum stable set problem. It is found that the proposed methods with expanded SD bases are significantly more efficient than methods using other approximations or solving semidefinite relaxation problems directly.

The organization of this paper is as follows. Various approximations of ${\cal S}^{n}_{+}$ are introduced in section 2, including those based on the factor width by Boman et al. [10], diagonal dominance by Ahmadi et al. [1], and SD bases by Tanaka and Yoshise [41]. The main results of this paper, i.e., an expansion of SD bases and an analysis of its theoretical properties, are provided in section 3. In section 4, we introduce the cutting plane method using different approximations of ${\cal S}^{n}_{+}$ for calculating upper bounds of the maximum stable set problem. We also describe the results of numerical experiments and evaluate the efficiency of the proposed method with expanded SD bases.

2 Some approximations of the semidefinite cone

2.1 Factor width approximation

In [10], Boman et al. defined a concept called factor width.

Definition 2.1**.**

(Definition 1 in [10])* The factor width of a real symmetric matrix $A\in\mathbb{S}^{n}$ is the smallest integer $k$ such that there exists a real matrix $V\in\mathbb{R}^{n\times m}$ where $A=VV^{T}$ and each column of $V$ contains at most $k$ nonzero elements.*

For $k\in\{1,2,\ldots,n\}$, we can also define

[TABLE]

It is obvious that the factor width is only defined for semidefinite matrices, because for every matrix $A$ in Definition 2.1, the decomposition $A=VV^{T}$ implies that $A\in{\cal S}^{n}_{+}$. Therefore, for every $k\in\{1,2,\ldots,n\}$, the set of matrices with a factor width of at most $k$ gives an inner approximation of $\mathbb{S}^{n}_{+}$: ${\cal FW}(k)\subseteq\mathbb{S}^{n}_{+}.$

2.2 Diagonal dominance approximation

In [1] and [2], the authors approximated the cone ${\cal S}^{n}_{+}$ with the set of diagonally dominant matrices and the set of scaled diagonally dominant matrices.

Definition 2.2**.**

The set of diagonally dominant matrices ${\cal DD}_{n}$ and the set of scaled diagonally dominant matrices ${\cal SDD}_{n}$ are defined as follows:

[TABLE]

It is easy to see that ${\cal DD}_{n}$ is a convex cone and ${\cal SDD}_{n}$ is a cone in $\mathbb{S}^{n}$. As a consequence of the Gershgorin circle theorem [18], we have the relation ${\cal DD}_{n}\subseteq{\cal SDD}_{n}\subseteq{\cal S}^{n}_{+}$. Ahmadi et al. [1] defined ${\cal U}_{n,k}$ as the set of vectors in $\mathbb{R}^{n}$ with at most $k$ nonzeros, each equal to $1$ or $-1$. They also defined a set of matrices $U_{n,k}:=\{uu^{T}\mid u\in{\cal U}_{n,k}\}$. Barker and Carlson [6] proved the following theorem.

Theorem 2.3**.**

(Barker and Carlson [6])* ${\cal DD}_{n}={\rm cone}(U_{n,2}).$*

The conical hull of a given set ${\cal K}\subseteq\mathbb{S}^{n}$ is defined as ${\rm cone}({\cal K}):=\{\sum_{i=1}^{k}\alpha_{i}X_{i}\mid X_{i}\in{\cal K},\alpha_{i}\geq 0,k\in{\mathbb{Z}}_{\geq 0}\}$, where $\mathbb{Z}_{\geq 0}$ is the set of nonnegative integers. A cone generated in this way by a finite number of elements is called finitely generated. Theorem 2.3 implies that ${\cal DD}_{n}$ has $n^{2}$ extreme rays; thus, it is a finitely generated cone.

A cone ${\cal K}\in\mathbb{S}^{n}$ is polyhedral if ${\cal K}=\{X\in\mathbb{S}^{n}\mid\langle A_{i},X\rangle\leq 0\}$ for some $A_{i}\in\mathbb{S}^{n}$. The following theorem follows from the results of Minkowski [34] and Weyl [45].

Theorem 2.4**.**

(Minkowski-Weyl theorem, see Corollary 7.1a in [39])* A convex cone is polyhedral if and only if it is finitely generated.*

The above theorem ensures that ${\cal DD}_{n}$ is a polyhedral cone. Using the expression in Theorem 2.3, Ahmadi et al. proved that optimization problems over ${\cal DD}_{n}$ can be solved as LPs. They also proved that optimization problems over ${\cal SDD}_{n}$ can be solved as SOCPs. They designed a column generation method using ${\cal DD}_{n}$ and ${\cal SDD}_{n}$ to obtain a series of inner approximations of ${\cal S}_{n}^{+}$. As for the relation between the factor width and diagonal dominance, useful results were presented in [10] and in [2], which gives a relation between ${\cal SDD}_{n}$ and the set of matrices with a factor width of at most $2$.

Lemma 2.5**.**

(See [10] and Theorem 8 in [2])* ${\cal FW}(2)={\cal SDD}_{n}$*

Note that Definition 2.1 implies that the set ${\cal FW}(k)$ is convex for any $k\in\{1,2,\ldots,n\}$, and we obtain the following corollary as Lemma 2.5:

Corollary 2.6**.**

The set ${\cal SDD}_{n}$ is a convex cone.

2.3 SD basis approximation

Tanaka and Yoshise defined semidefinite (SD) bases [41].

Definition 2.7**.**

(Definitions 1 and 2 in [41])* Let $e_{i}\in\mathbb{R}^{n}$ denotes the vector with a $1$ at the $i$th coordinate and [math] elsewhere, and let $I=(e_{1},\ldots,e_{n})\in\mathbb{S}^{n}$ be the identity matrix. Then*

[TABLE]

is called an SD basis of Type I, and

[TABLE]

is called an SD basis of Type II. Matrices in SD bases Type I and II are defined as

[TABLE]

As shown in [41], ${\cal B}_{+}$ and ${\cal B}_{-}$ are subsets of $\mathcal{S}^{n}_{+}$ and bases of $\mathbb{S}^{n}$. Given a set ${\cal K}\subseteq\mathbb{S}^{n}$, we define the dual cone of ${\cal K}$ as $({\cal K})^{*}:=\{A\in\mathbb{S}^{n}\mid\langle A,B\rangle\geq 0\ \mbox{for any}\ B\in{\cal K}\}$. The conical hull of ${\cal B}_{+}\cup{\cal B}_{-}$ and its dual give an inner and an outer polyhedral approximation of $\mathcal{S}^{n}_{+}$, as follows.

Definition 2.8**.**

Let $I=(e_{1},\ldots,e_{n})\in\mathbb{S}^{n}$ be the identity matrix. The inner and outer approximations of ${\cal S}^{n}_{+}$ by using SD bases are defined as

[TABLE]

By Definition 2.7, we know that ${\cal B}_{+},{\cal B}_{-}\subseteq{\cal S}^{n}_{+}$. Since ${\cal S}^{n}_{+}$ is a convex cone, we have ${\cal S}_{\rm in}\subseteq{\rm cone}({\cal S}^{n}_{+})={\cal S}^{n}_{+}$. By Lemma 1.7.3 in [32], we know that ${\cal S}^{n}_{+}$ is self-dual; that is, ${\cal S}^{n}_{+}=({\cal S}^{n}_{+})^{*}$. Accordingly, we can conclude that ${\cal S}_{\rm in}\subseteq{\cal S}^{n}_{+}\subseteq{\cal S}_{\rm out}$.

Remark 2.9**.**

In [41], ${\cal B}_{+}$ and ${\cal B}_{-}$ are defined as ${\cal B}_{+}(P)$ and ${\cal B}_{-}(P)$ using an orthogonal matrix $P$ instead of the identity matrix $I$. In fact, for any orthogonal matrix $P$,

[TABLE]

also give other bases and generalizations of ${\cal B_{+}}$ and ${\cal B_{-}}$. However, as we will see in section 4, we use the matrices in the bases as in optimization problems of the form

[TABLE]

which is equivalent to

[TABLE]

Therefore, we consider that the generalizations $P{\cal B_{+}}P^{T}$ and $P{\cal B_{-}}P^{T}$ are not essential throughout this paper and omit those descriptions from subsequent sections to simplify the presentation.

3 Expansion of SD bases

When we use the SD bases for approximating $\mathcal{S}^{n}_{+}$, the sparsity of the matrices in those bases is quite important in terms of computational efficiency. As we mentioned in Remark 2.9, for any orthogonal matrix $P$, $P{\cal B_{+}}P^{T}$ and $P{\cal B_{-}}P^{T}$ give generalizations of the SD bases. However, it is hard to choose an appropriate orthogonal matrix $P$ (except for the identity matrix $I$) to keep the sparsity of the matrices $PCP^{T}$ and $PAP^{T}$ in (2). In this section, we try to extend the definition of the SD bases in order to obtain various sparse SD bases which will lead us to sparse polyhedral approximations of $\mathcal{S}^{n}_{+}$.

3.1 SD bases and their relations with ${\cal S}^{n}_{+}$ and ${\cal DD}_{n}$

First, we give a lemma that provides an expression of ${\cal S}^{n}_{+}$ by using SD bases. The lemma is a direct corollary of the fact that any $X\in{\cal S}^{n}_{+}$ has nonnegative eigenvalues and a corresponding orthogonal basis of eigenvectors.

Lemma 3.1**.**

[TABLE]

where ${\cal O}^{n}$ is the set of orthogonal matrices in $\mathbb{R}^{n\times n}$.

Lemma 3.1 gives a way to approximate ${\cal S}^{n}_{+}$ by changing the matrix $P=(p_{1},..,p_{n})$ $\in{\cal O}^{n}$ when creating SD bases. However, a dense matrix $P\in{\cal O}^{n}$ may lead to a dense formulation of the approximation using SD basis, which is unattractive from the standpoint of computational efficiency.

Note that we can easily see that the set ${\rm cone}({\cal B}_{+}\cup{\cal B}_{-})$, the conical hull of the sparse SD bases $\mathcal{B}_{+}$ and $\mathcal{B}_{-}$, is equivalent to ${\rm cone}(U_{n,2})$. Thus, we obtain the following proposition as a corollary of Theorem 2.3.

Proposition 3.2**.**

[TABLE]

3.2 Expansion of SD bases without losing sparsity

The previous section shows that we can obtain a sparse polyhedral approximation of $\mathcal{S}^{n}_{+}$ by using the SD bases. In this section, we try to extend the definition of the SD bases in order to obtain various sparse polyhedral approximations of $\mathcal{S}^{n}_{+}$.

Definition 3.3**.**

Let $I=(e_{1},\ldots,e_{n})\in\mathbb{S}^{n}$ be the identity matrix. Define the expansion of the SD basis with one parameter $\alpha\in\mathbb{R}$ as

[TABLE]

The proposition below ensures that the expansion of the SD bases also gives bases of $\mathbb{S}^{n}$.

Proposition 3.4**.**

Let $I=(e_{1},\ldots,e_{n})\in\mathbb{S}^{n}$ be the identity matrix. For any $\alpha\in\mathbb{R}\setminus\{0,-1\}$, $\bar{\cal B}(\alpha)$ is a set of $n(n+1)/2$ independent matrices and thus a basis of $\mathbb{S}^{n}$.

Proof.

Let $\alpha\in\mathbb{R}\setminus\{0,-1\}$. Accordingly, for $1\leq i<j\leq n$, we have

[TABLE]

and for every $1\leq i\leq n$, we also have

[TABLE]

Suppose that there exist $\gamma_{i,j}\geq 0\ (1\leq i\leq j\leq n)$ such that

[TABLE]

Then, by (3) and (4), we see that

[TABLE]

Since $\{B_{i,j}^{+}\}={\cal B}_{+}$ is a set of linearly independent matrices, all the coefficients for ${B}_{i,j}$ in (5) should be [math]. Thus, we have

[TABLE]

Since $\alpha\neq 0$, by (9) we have

[TABLE]

Since $\alpha\neq-1$, (6)-(10) imply that

[TABLE]

The above leads us to conclude that $\{\bar{B}_{i,j}(\alpha)\}=\bar{\cal B}(\alpha)$ is a set of $n(n+1)/2$ linearly independent matrices. $\square$ ∎

If we let $\alpha=1$, then it is straightforward that $\bar{\cal B}(1)={\cal B}_{+}$. If we let $\alpha$ be other real numbers, we may obtain different SD bases. The following proposition gives the condition for generating different expanded SD bases.

Proposition 3.5**.**

Let $I=(e_{1},\ldots,e_{n})\in\mathbb{S}^{n}$ be the identity matrix. Suppose that $\alpha_{1}\in\mathbb{R}\setminus\{0,-1\}$ and $\alpha_{2}\in\mathbb{R}\setminus\{0,\alpha_{1}\}$. Then, for every $1\leq i<j\leq n$,

[TABLE]

Proof.

For $1\leq i\leq j\leq n$, let us define

[TABLE]

Note that if $i=j$, then

[TABLE]

For every $i<j$, we can write $\bar{B}_{i,j}^{2}$ as a linear combination of $\bar{B}_{i,j}^{1}$:

[TABLE]

Since $\alpha_{1}\not\in\{0,-1\}$, Proposition 3.4 ensures that $\mathcal{\bar{B}}(\alpha_{1})$ is linearly independent, and hence, the expression (13) for ${\bar{B}}_{i,j}^{2}$ is unique.

Suppose that $\bar{B}_{i,j}^{2}\in{\rm cone}\left(\bar{\cal B}(\alpha_{1})\right)$. In this case, all the coefficients in (13) should be nonnegative, which implies that

[TABLE]

From the last inequality in (14), we have either

[TABLE]

For case (i), from the first and second inequalities of (14), we have $\alpha_{2}-\alpha_{1}\geq 0$ and $\alpha_{1}-\alpha_{2}\geq 0$, which implies $\alpha_{2}=\alpha_{1}$ and contradicts the assumption $\alpha_{2}\neq\alpha_{1}$. A similar contradiction is obtained for case (ii). Thus, we have $\bar{B}_{i,j}^{2}\notin{\rm cone}(\bar{\cal B}(\alpha_{1}))$. $\square$ ∎

3.3 Expression of ${\cal SDD}_{n}$ with expanded SD bases

As we have seen in Corollary 2.6, the set ${\cal SDD}_{n}={\cal FW}(2)$ is a convex cone. This fact ensures that as a corollary of Theorem 2.3, the conical hull of the union of the extended SD bases $\bar{\cal B}(\alpha)$ on $\alpha\in\mathbb{R}$ coincides with ${\cal FW}(2)$ and hence, the set of scaled diagonally dominant matrices ${\cal SDD}_{n}$:

Corollary 3.6**.**

[TABLE]

3.4 Notes on the parameter $\alpha$

Here, we discuss the choice for the parameter $\alpha$ to increase the “volume” of the polyhedral approximation ${\rm cone}(\bar{\cal B}(\alpha))$ of the semidefinite cone ${\cal S}^{n}_{+}$. For any $\alpha\in\mathbb{R}$ and $1\leq i<j\leq n$, by Definition 3.3, we can calculate the Frobenius norm of $\bar{B}_{i,j}(\alpha)$:

[TABLE]

According to Proposition 3.5, by changing $\alpha$, one can obtain different polyhedral approximations. However, we can see that

[TABLE]

and by Definitions 2.7 and 3.3, we have

[TABLE]

This shows that, if $|\alpha|\rightarrow\infty$ or $\alpha\in\{0,1,-1\}$, the new matrix $\bar{B}_{i,j}(\alpha)$ will become close to the existing matrices, e.g. ${B}^{+}_{i,i}$, ${B}^{+}_{j,j}$, ${B}^{+}_{i,j}$ and ${B}^{-}_{i,j}$, and the “volume” of the polyhedral approximation ${\rm cone}(\bar{\cal B}(\alpha)\cup{\cal B}_{+}\cup{\cal B}_{-})$ of the semidefinite cone ${\cal S}^{n}_{+}$ will also be close to the “volume” of the existing inner approximation ${\rm cone}({\cal B}_{+}\cup{\cal B}_{-})$ of ${\cal S}^{n}_{+}$.

To give an illustrative explanation of the above discussion, here we consider the specific case

[TABLE]

and draw some figures in $\mathbb{R}^{3}$ with coordinate $a,b$ and $c$. Fig. 1 [a] shows the set of ${\cal S}^{2}_{+}$ in $\mathbb{R}^{3}$. The red arrow in Fig. 1 [b] shows the extreme rays $\{\gamma\bar{B}_{i,j}(\alpha)\mid\gamma\geq 0\}$ with $|\alpha|\rightarrow\infty$ and $\alpha\in\{0,1,-1\}$. The conical hull of these extreme rays is ${\rm cone}({\cal B}_{+}\cup{\cal B}_{-})$ and its cross section with $\{X\in\mathbb{S}^{2}\mid\langle X,I\rangle=1\}$ is illustrated as the blue area. To avoid generating a new matrix $\bar{B}_{i,j}(\alpha)$ that is close to the existing matrices, we should choose an $\alpha$ such that the angle between $\bar{B}_{i,j}(\alpha)$ and existing matrices are equal, as illustrated in Fig. 1 [c].

We expand this idea to the case of generating a matrix $\bar{B}_{i,j}(\alpha)\in\mathbb{S}^{n}$. Given an $\alpha\in\mathbb{R}$, we can define the angles between matrices in the expanded SD bases and SD bases Type I and II for every $1\leq i<j\leq n$, as follows:

[TABLE]

Thus, we have

[TABLE]

Similarly, we have

[TABLE]

In general, to obtain a large enough inner approximation with limited parameters, we prefer an $\alpha$ that makes $\theta_{1}(\alpha)=\theta_{3}(\alpha)$, which means that the new matrix $\bar{B}_{i,j}(\alpha)$ will be in the middle of ${B}^{+}_{i,i}$ and ${B}^{+}_{i,j}$ on the boundary of ${\cal S}^{n}_{+}$. Similarly, we can obtain $\alpha$ by calculating $\theta_{2}(\alpha)=\theta_{3}(\alpha)$, $\theta_{1}(\alpha)=\theta_{4}(\alpha)$ and $\theta_{2}(\alpha)=\theta_{4}(\alpha)$. By solving these equalities, we find that

[TABLE]

The expansions with these parameters are expected to provide generally large inner approximations for ${\cal S}^{n}_{+}$.

4 Cutting plane methods for the maximum stable set problem

Conic optimization problems, including SDPs and copositive programs, have been shown to provide tight bounds for NP-hard combinatorial and noconvex optimization problems. Here, we consider applying approximations of ${\cal S}^{n}_{+}$ to one of those NP-hard problems, the maximum stable set problem. A stable set of a graph $G(V,E)$ is a set of vertices in $V$, such that there is no edge connecting any pair of vertices in the set. The maximum stable set problem aims to find the stability number, i.e. the number of vertices of the largest stable set of $G$, namely $\alpha(G)$.

De Klerk and Pasechnik [15] proposed a copositive programming formulation to obtain the exact stability number of a graph $G$ with $n$ vertices:

[TABLE]

where $e$ is the all-ones vector, $A$ is the adjacency matrix of graph $G$, and ${\cal C}^{*}_{n}$ is the dual cone of the copositive cone ${\cal C}_{n}:=\{X\in\mathbb{S}^{n}\mid d^{T}Xd\geq 0\ \forall d\in\mathbb{R}^{n},\ d\geq 0\}$.

Although problem (4) is a conic optimization problem, it is still difficult since determining whether $X\in{\cal C}^{*}_{n}$ or not is NP-hard [16]. A natural approach is to relax this problem to a more tractable optimization problem. From the definition of each cone, we can see the validity of the following inclusions:

[TABLE]

By replacing ${\cal C}^{*}_{n}$ with ${\cal S}^{n}_{+}\cap{\cal N}^{n}$, one can obtain an SDP relaxation of (4):

[TABLE]

Solving this SDP is not as easy as it seems to be; in fact, we could not obtain a useful result of (4) after 6 hours of calculation using the state-of-the-art SDP solver Mosek for a random generalized problem when $n=300$. Combining the expanded SD bases with the cutting plane method, we apply the approximations of ${\cal S}^{n}_{+}$ to (4) and solve it by calculating a series of more tractable problems.

Let ${\cal P}^{n}$ satisfy ${\cal S}^{n}_{+}\subseteq{\cal P}^{n}\subseteq\mathbb{S}^{n}$ and replace $X\in{\cal S}^{n}_{+}$ by $X\in{\cal P}^{n}$ in (4). Then, we obtain a relaxation of (4):

[TABLE]

Usually, the relaxed problem (4) is expected to be easier to solve and to give us a better upper bound of problem (4) from its optimal solution $X^{*}$. To get a better upper bound, we select some eigenvectors with negative eigenvalues of an optimal solution $X^{*}$ of problem (4), say $d_{1},..,d_{k}$, by adding cutting planes

[TABLE]

to (4), and obtain a new optimization problem

[TABLE]

Notice that the optimal solution $X^{*}$ of problem (4) is cut from the feasible region of problem (4) since $\langle d_{i}d_{i}^{T},X^{*}\rangle<0\ (i=1,..,k)$. On the other hand, since ${\cal S}^{n}_{+}=\{X\in\mathbb{S}^{n}\mid\forall d\in\mathbb{R}^{n},\ \langle dd^{T},X\rangle\geq 0\}\subseteq{\cal P}^{n}$, every feasible solution of (4) is feasible for (4), and hence problem (4) is a relaxation of problem (4). These facts ensure that problem (4) is a tighter relaxation of problem (4) than problem (4). By repeating this procedure, we are able to obtain a series of nonincreasing upper bounds of (4). Since the eigenvectors are usually dense, we only have to add eigenvectors corresponding to up to the second smallest eigenvalues to $\{d_{i}\}$ at every iteration, which increases computational efficiency.

As for the selection of the initial relaxation ${\cal P}^{n}$, we are ready to use the approximations of ${\cal S}^{n}_{+}$ based on the expanded SD bases. Let ${\cal H}:=\{\pm 1,\pm 1\pm\sqrt{2}\}$ be the set of parameters calculated in Section 3.4, and let ${\cal SDB}_{n}$ denote the conical hull of expanded SD bases using ${\cal H}$:

[TABLE]

Then, as has been described in the previous sections, we have

[TABLE]

If ${\cal SDB}^{*}_{n}$ or ${\cal DD}^{*}_{n}$ is selected to be ${\cal P}_{n}$, the corresponding relaxed problem in the cutting plane procedure becomes an LP, which allows us to use powerful state-of-the-art LP solvers, such as Gurobi [21]. Ahmadi et. al. [1] showed that when ${\cal SDD}^{*}_{n}$ is selected, the relaxations turn out to be SOCPs. Although ${\cal SDD}^{*}_{n}$ provides a tighter relaxation than either ${\cal DD}_{n}$ or ${\cal SDB}_{n}$, the latter two relaxations are expected to have a lower computational cost. In addition, in [1], Ahmadi et al. also proposed an SOCP-based cutting plane approach, named SDSOS, which adds SOCP cuts at every iteration. We conducted experiments to compare the efficiencies of those cutting plane methods using different approximations and SDSOS. The specifications of the experimental methods are summarized in Table 1.

We tested these methods on the Erd$\ddot{\rm o}$s-Rényi graphs $ER(n,p)$, randomly generated by Ahmadi et al. in [1], where $n$ is the number of vertices and every pair of vertices has an edge with probability $p$. All experiments were performed with MATLAB 2018b on a Windows PC with an Intel(R) Core(TM) i7-6700 CPU running at 3.4 GHz and 16 GB of RAM. The LPs were solved using Gurobi Optimizer 8.0.0 [21] and the SOCPs and SDPs are solved using Mosek Optimizer 9.0 [35].

Fig. 2 shows the result for an instance with $n=250$ and $p=0.8$. The x-axis is the number of iterations, and the y-axis is the gap between the upper bounds of each method and the SDP bound obtained by (4); the gap is computed by $\left|\frac{f^{*}-f_{k}}{f^{*}}\right|\times 100\%$ for the obtained upper bound $f_{k}$ at $k$’s iteration and the SDP bound $f^{*}$ obtained by solving (4) directly.

As can be seen in this figure, the accuracy of CPDD is the worst among the four methods at each iteration. CPSDB achieves almost the same upper bounds as CPSDD and SDSOS, which shows that the proposed polyhedral approximation ${\cal SDB}_{n}$ is promising for obtaining a solution close to the non-polyhedral approximation ${\cal SDD}_{n}$ of ${\cal S}^{n}_{+}$. Although SDSOS adds an extra SOCP cut at every iteration and takes longer to solve, the accuracy of SDSOS does not seem to be affected and is not so different from the accuracy of CPSDD at each iteration.

Fig. 3 shows the relation between the computation time and the gap of each method for the same instance. Although its accuracy is not necessarily the best at every iteration, it seems that CPSDB is the most efficient method. CPSDB attains an upper bound whose gap is $2$ within $30$ s, while CPSDD and SDSOS attain upper bounds whose gap is $4$ after the same amount of time. The difference might come from that the subproblems of CPSDB are sparse LPs at earlier iterations and the computations are relatively cheaper than those of CPSDD and SDSOS whose subproblems are SOCPs.

Table 2 and 3 give the bounds of iterative methods and the SDP bound for all the instances. In Table 2, the CPSDD0/SDSOS0 column shows the first upper bound obtained by CPSDD and SDSOS, i.e., the upper bound obtained by solving the same SOCP before adding any cutting plane. The (5 min) and (10 min) columns of CPSDD (SDSOS) show the upper bounds obtained after $5$ minutes and after $10$ minutes of the CPSDD (SDSOS) computation, respectively. The SDP column shows the SDP bound obtained by solving (4).

Similarly, in Table 3, the CPDD0 and CPSDB0 columns show the first upper bounds obtained by CPDD and CPSDB, respectively, before adding any cutting plane. The (5 min) and (10 min) columns of CPDD (CPSDB) show the upper bounds obtained after $5$ minutes and after $10$ minutes of the CPDD (CPSDB) computation, respectively.

Note that we failed to solve SDPs (4) for instances having $n=300$ nodes within our time limit $20000s$. In Table 2, the Value and Time (s) columns of SDP with $n=300$ show the results obtained in [1] for these two instances, as a reference.

As can be seen in Table 2 and 3, for all instances, the values of CPSDD0/SDSOS0 are better than the values of CPSDB0 and CPDD0. These results correspond to the inclusion relationship of initial approximations (20). We can also see that the values of CPSDB0 are almost the same as those of CPSDD0/SDSOS0 for all instances, while the values of CPDD0 are much worse than others. For all instances, CPSDB seems to be significantly more efficient than all other methods. For example, for instance with $n=250$ and $p=0.3$, after $10$ min of calculation, CPSDB obtained an upper bound of $73.24$, while CPSDD and SDSOS got upper bounds greater than $90$ and CPDD got a bound of more than $146$.

At present, solving a large SDP, e.g., one with more than $n=300$ nodes requires a significant amount of computational time. The cutting plane method CPSDB with our polyhedral approximation ${\cal SDB}_{n}$ is a promising way of obtaining efficient upper bounds of such large SDPs in a moderate time.

5 Concluding remarks

We developed techniques to construct a series of sparse polyhedral approximations of the semidefinite cone. We provided a way to approximate the semidefinite cone by using SD bases and proved that the set of diagonally dominant matrices can be expressed with sparse SD bases. We proposed a simple expansion of SD bases that keeps the sparsity of the matrices that compose it. We gave the conditions for generating linearly independent matrices in expanded SD bases as well as for generating an expansion different from the existing one. We showed that the polyhedral approximation using our expanded SD bases contains the set of diagonally dominant matrices and is contained in the set of scaled diagonally dominant matrices. We also proved that the set of scaled diagonally dominant matrices can be expressed using an infinite number of expanded SD bases.

The polyhedral approximations were applied to the cutting plane method for solving a semidefinite relaxation of the maximum stable set problem. The results of the numerical experiments showed that the method with our expanded SD bases is more efficient than other methods (see Fig. 3); improving the efficiency of our method still remains an important study issue.

One future direction of study is to increase the number of vectors in the definition of the SD bases. The current SD bases are defined as a set of matrices $(e_{i}+e_{j})(e_{i}+e_{j})^{T}$. If we use three vectors, as in $(e_{i}+e_{j}+e_{k})(e_{i}+e_{j}+e_{k})^{T}$, we might obtain another inner approximation that remains relatively sparse when the dimension $n$ is large.

Another future direction is to focus on the factor width $k$ of a matrix. The cone of matrices with factor width at most $k=2$ was introduced in order to give another expression of the set $\mathcal{SDD}_{n}$ of scaled diagonally dominant matrices. By considering a larger width $k>2$, we may obtain a larger inner approximation of the semidefinite cone $\mathcal{S}^{n}_{+}$, although it would not be polyhedral, or even characterized by using SOCP constraints. Finding efficient ways to solve approximation problems over such cones might be an interesting challenge.

Also, our expanded SD bases can be applied to some other difficult problems. Mixed integer nonlinear programming has recently become popular in many practical applications. In [33], Lubin et al. proposed a cutting plane framework for mixed integer convex optimization problems. In [26], Kobayashi and Takano proposed a branch and bound cutting plane method for mixed integer SDPs. It would be interesting to see whether the approximations of ${\cal S}^{n}_{+}$ proposed in this paper could be used to improve the efficiency of those methods.

Acknowledgments

The authors would like to sincerely thank the anonymous reviewers for their thoughtful and valuable comments which have significantly improved the paper. Among others, one of the reviewers pointed out Remark 2.9 which helped the authors to simplify the presentation of the paper. This research was supported by the Japan Society for the Promotion of Science through a Grant-in-Aid for Challenging Exploratory Research (17K18946) and a Grant-in-Aid for Scientific Research ((B)19H02373) from the Ministry of Education, Culture, Sports, Science and Technology of Japan.