Extension of the LP-Newton method to SOCPs via semi-infinite representation

TL;DR

This paper extends the LP-Newton method to solve second-order cone programming problems by reformulating them as semi-infinite programs and demonstrates its convergence and efficiency through numerical experiments.

Contribution

The paper introduces a novel extension of the LP-Newton method to SOCPs using semi-infinite programming reformulation, with proven convergence and practical efficiency.

Findings

The extended method converges globally under mild conditions.

Numerical experiments show competitive efficiency with interior-point methods.

The approach provides a new perspective for solving SOCPs via projection methods.

Abstract

The LP-Newton method solves the linear programming problem (LP) by repeatedly projecting a current point onto a certain relevant polytope. In this paper, we extend the algorithmic framework of the LP-Newton method to the second-order cone programming problem (SOCP) via a linear semi-infinite programming (LSIP) reformulation of the given SOCP. In the extension, we produce a sequence by projection onto polyhedral cones constructed from LPs obtained by finitely relaxing the LSIP. We show the global convergence property of the proposed algorithm under mild assumptions, and investigate its efficiency through numerical experiments comparing the proposed approach with the primal-dual interior-point method for the SOCP.

Tables

Table 1. Table 1: Performance of the Adaptive LP-Newton method.

# dimensions					# hyperplanes
$m$	$n$	$(n_1,n_2,\dots ,n_p)$	time [s]	# iter	initial	final
$10$	$200$	$(1,1,\dots ,1)$	0.1	3.5	200.0	200.0
$10$	$200$	$(2,2,\dots ,2)$	0.3	3.8	200.0	385.5
$10$	$200$	$(5,5,\dots ,5)$	7.3	19.1	320.0	1019.4
$10$	$200$	$(10,10,\dots ,10)$	58.4	62.5	360.0	1580.9
$10$	$200$	$(20,20,\dots ,20)$	167.6	141.1	380.0	1779.4
$10$	$200$	$(100,100)$	136.8	352.0	396.0	1098.0
$10$	$200$	$200$	48.1	274.4	398.0	671.4
$10$	$350$	$(1,1,\dots ,1)$	0.1	3.6	350.0	350.0
$10$	$350$	$(2,2,\dots ,2)$	0.9	3.5	350.0	592.8
$10$	$350$	$(5,5,\dots ,5)$	34.0	19.2	560.0	1786.6
$10$	$350$	$(10,10,\dots ,10)$	303.0	64.7	630.0	2849.2
$10$	$350$	$(35,35,\dots ,35)$	1911.0	281.9	680.0	3489.0
$10$	$350$	$(175,175)$	462.4	414.7	696.0	1523.4
$10$	$350$	$350$	104.4	217.3	698.0	914.3
$10$	$500$	$(1,1,\dots ,1)$	0.1	3.8	500.0	500.0
$10$	$500$	$(2,2,\dots ,2)$	2.0	3.5	500.0	805.7
$10$	$500$	$(5,5,\dots ,5)$	98.0	19.9	800.0	2638.5
$10$	$500$	$(10,10,\dots ,10)$	856.6	66.0	900.0	4135.5
$10$	$500$	$(50,50,\dots ,50)$	5618.0	372.5	980.0	4694.8
$10$	$500$	$(250,250)$	866.0	398.1	996.0	1790.1
$10$	$500$	$500$	183.6	172.7	998.0	1169.7
$50$	$200$	$(1,1,\dots ,1)$	0.0	4.0	200.0	200.0
$50$	$200$	$(2,2,\dots ,2)$	0.3	4.0	200.0	461.8
$50$	$200$	$(5,5,\dots ,5)$	5.2	16.8	320.0	947.6
$50$	$200$	$(10,10,\dots ,10)$	26.3	47.9	360.0	1297.9
$50$	$200$	$(20,20,\dots ,20)$	67.3	105.3	380.0	1423.0
$50$	$200$	$(100,100)$	62.7	257.1	396.0	908.2
$50$	$200$	$200$	30.5	238.1	398.0	635.1
$50$	$350$	$(1,1,\dots ,1)$	0.1	4.0	350.0	350.0
$50$	$350$	$(2,2,\dots ,2)$	1.3	3.9	350.0	752.3
$50$	$350$	$(5,5,\dots ,5)$	27.3	17.2	560.0	1690.5
$50$	$350$	$(10,10,\dots ,10)$	184.6	54.4	630.0	2499.0
$50$	$350$	$(35,35,\dots ,35)$	675.8	200.5	680.0	2674.9
$50$	$350$	$(175,175)$	235.7	308.3	696.0	1310.6
$50$	$350$	$350$	64.0	164.6	698.0	861.6
$50$	$500$	$(1,1,\dots ,1)$	0.1	3.9	500.0	500.0
$50$	$500$	$(2,2,\dots ,2)$	3.2	3.8	500.0	1026.4
$50$	$500$	$(5,5,\dots ,5)$	80.1	17.9	800.0	2487.1
$50$	$500$	$(10,10,\dots ,10)$	548.5	56.4	900.0	3669.7
$50$	$500$	$(50,50,\dots ,50)$	2287.4	274.9	980.0	3719.0
$50$	$500$	$(250,250)$	392.6	281.0	996.0	1556.0
$50$	$500$	$500$	125.5	140.3	998.0	1137.3
$100$	$200$	$(1,1,\dots ,1)$	0.0	4.0	200.0	200.0
$100$	$200$	$(2,2,\dots ,2)$	0.3	4.0	200.0	481.9
$100$	$200$	$(5,5,\dots ,5)$	3.8	14.9	320.0	875.7
$100$	$200$	$(10,10,\dots ,10)$	13.6	38.1	360.0	1102.0
$100$	$200$	$(20,20,\dots ,20)$	30.3	78.4	380.0	1154.0
$100$	$200$	$(100,100)$	35.3	195.4	396.0	784.8
$100$	$200$	$200$	24.5	211.5	398.0	608.5
$100$	$350$	$(1,1,\dots ,1)$	0.1	4.0	350.0	350.0
$100$	$350$	$(2,2,\dots ,2)$	1.5	4.0	350.0	828.4
$100$	$350$	$(5,5,\dots ,5)$	24.1	16.6	560.0	1645.8
$100$	$350$	$(10,10,\dots ,10)$	114.6	46.9	630.0	2236.5
$100$	$350$	$(35,35,\dots ,35)$	397.4	166.1	680.0	2331.0
$100$	$350$	$(175,175)$	159.0	257.8	696.0	1209.6
$100$	$350$	$350$	87.9	212.3	698.0	909.3
$100$	$500$	$(1,1,\dots ,1)$	0.1	4.0	500.0	500.0
$100$	$500$	$(2,2,\dots ,2)$	5.9	4.0	500.0	1195.9
$100$	$500$	$(5,5,\dots ,5)$	72.2	17.0	800.0	2399.6
$100$	$500$	$(10,10,\dots ,10)$	388.0	50.2	900.0	3360.0
$100$	$500$	$(50,50,\dots ,50)$	1286.0	223.6	980.0	3206.0
$100$	$500$	$(250,250)$	399.2	290.1	996.0	1574.2
$100$	$500$	$500$	120.4	139.4	998.0	1136.4

Table 2. Table 2: Comparison with the primal-dual interior-point method.

# dimensions			time [s]
$m$	$n$	$(n_1,n_2,\dots ,n_p)$	ALPN	SDPT3
$1400$	$1500$	$(3,3,\dots ,3)$	177.3	366.6
$1700$	$1800$	$(3,3,\dots ,3)$	260.4	638.4
$2000$	$2100$	$(3,3,\dots ,3)$	363.4	970.0

Equations

$$\begin{array}[]{ll}\text{maximize}&c^{\top}x\\ \text{subject to}&Ax=b,\\ &x\in\mathcal{K},\end{array}$$

$${\mathcal{K}}^l:=\{\begin{array}{ll}\left\{(z_1,z_2,\dots ,z_l)\in {\mathbb{R}}^l:z_1\ge \sqrt{\sum _{j=2}^l{z}_j^2}\right\}& (l\ge 2),\\ \{z\in \mathbb{R}:z\ge 0\}& (l=1).\end{array}$$

$$\begin{array}[]{ll}\text{maximize}&c^{\top}x\\ \text{subject to}&Ax=b,\\ &x\geq 0.\end{array}$$

$$\begin{array}[]{lll}\text{maximize}&c^{\top}x\\ \text{subject to}&Ax=b,\\ &(1,(v^{i})^{\top})x^{i}\geq 0&(v^{i}\in V_{i};\ i=1,2,\dots,p),\end{array}$$

$$\bar{A}:=\left(\begin{array}{c}{c}^{\top }\\ A\end{array}\right)\in {\mathbb{R}}^{(1+m)\times n},L:=\left\{\left(\begin{array}{c}\gamma \\ b\end{array}\right):\gamma \in \mathbb{R}\right\},$$

$$E:=\prod _{i=1}^p{E}_i,{\mathcal{K}}_E:=\{x\in {\mathbb{R}}^n:(1,(v^i{)}^{\top })x^i\ge 0(\forall v^i\in E_i;i=1,2,\dots ,p)\}.$$

$$v^{i,(k)}\in \underset{v^i\in V_i}{\mathrm{argmin}}(1,(v^i{)}^{\top })x^{i,(k)}$$

$$\begin{array}[]{lll}\text{minimize}&\|\bar{A}x-w^{(k)}\|^{2}\\ \text{subject to}&(1,(v^{i})^{\top})x^{i}\geq 0&(\forall v^{i}\in E_{i}^{(k)};i=1,2,\dots,p),\end{array}$$

$${\gamma }^{(k+1)}={\zeta }^{(k)}-\frac{\Vert b-b^{(k)}{\Vert }^2}{{\gamma }^{(k)}-{\zeta }^{(k)}}.$$

$$v^{i,(k)}=-\frac{{\bar{x}}^{i,(k)}}{\Vert {\bar{x}}^{i,(k)}\Vert }$$

$$\begin{array}[]{ll}\text{minimize}&b^{\top}y\\ \text{subject to}&A^{\top}y-c\in\mathcal{K}.\end{array}$$

$${\theta }^{\ast }\le {\gamma }^{(k+1)}\le {\zeta }^{(k)}\le {\gamma }^{(k)}.$$

$${\theta }^{(k)}\le {\gamma }^{(k+1)}\le {\zeta }^{(k)}\le {\gamma }^{(k)}.$$

$$|{\gamma }^{(k)}-{\gamma }^{(k+1)}|=\Vert w^{(k)}-w^{(k+1)}\Vert \ge \Vert w^{(k)}-{\bar{w}}^{(k)}\Vert =\Vert \left(\begin{array}{c}{\gamma }^{(k+1)}-{\zeta }^{(k)}\\ b^{(k)}-b\end{array}\right)\Vert .$$

$$\frac{A{x}^{(k)}}{\Vert x^{(k)}\Vert }=\frac{b^{(k)}}{\Vert x^{(k)}\Vert },\frac{x^{(k)}}{\Vert x^{(k)}\Vert }\in {\mathcal{K}}_{E^{(k)}},\frac{c^{\top }{x}^{(k)}}{\Vert x^{(k)}\Vert }\le \frac{{\gamma }^{(0)}}{\Vert x^{(k)}\Vert },$$

$$A{d}^{\ast }=0,d^{\ast }\in \mathcal{K},c^{\top }{d}^{\ast }\le 0,\Vert d^{\ast }\Vert =1,$$

$$A(\bar{z}+s{d}^{\ast })=b,\bar{z}+s{d}^{\ast }\in \mathcal{K},c^{\top }(\bar{z}+s{d}^{\ast })\le c^{\top }\bar{z}$$

$$\begin{array}[]{ll}\text{minimize}&b^{\top}y\\ \text{subject to}&A^{\top}y-c\in\mathcal{K}.\end{array}$$

$$A{x}^{\ast }=b,-c+A^{\top }{y}^{\ast }-\eta =0,\eta \in \mathcal{K},x^{\ast }\in \mathcal{K},{\eta }^{\top }{x}^{\ast }=0.$$

$$\bar{A}{x}^{\ast }\in \underset{w\in \bar{A}\mathcal{K}}{\mathrm{argmax}}(1,-(y^{\ast }{)}^{\top })w.$$

$$x^{\ast }\in \underset{x\in \mathcal{K}}{\mathrm{argmax}}(1,-(y^{\ast }{)}^{\top })\left(\begin{array}{c}{c}^{\top }x\\ Ax\end{array}\right).$$

$$-c+A^{\top }{y}^{\ast }-\eta =0,\eta \in \mathcal{K},x^{\ast }\in \mathcal{K},{\eta }^{\top }{x}^{\ast }=0,$$

$$y^{(k)}=-\frac{b-A{x}^{(k)}}{{\gamma }^{(k)}-c^{\top }{x}^{(k)}}$$

$$\bar{A}{x}^{(k)}\in \underset{w\in \bar{A}{\mathcal{K}}_{E^{(k)}}}{\mathrm{argmax}}(1,-(y^{(k)}{)}^{\top }{)}^{\top }w,$$

$$x^{(k)}\in \underset{x\in {\mathcal{K}}_{E^{(k)}}}{\mathrm{argmax}}{c}^{\top }x-(y^{(k)}{)}^{\top }Ax.$$

$$-c+A^{\top }{y}^{(k)}\in {\mathcal{K}}_{E^{(k)}}^{\ast },x^{(k)}\in {\mathcal{K}}_{E^{(k)}},(-c+A^{\top }{y}^{(k)}{)}^{\top }{x}^{(k)}=0,A{x}^{(k)}=b^{(k)},$$

$$A^{\top }{d}^{\ast }\in \mathcal{K},x^{\ast }\in \mathcal{K},(A{x}^{\ast }{)}^{\top }{d}^{\ast }=0,A{x}^{\ast }=b.$$

$$b^{\top }(\bar{y}+s{d}^{\ast })=b^{\top }\bar{y}+s(A{x}^{\ast }{)}^{\top }{d}^{\ast }=b^{\top }\bar{y},$$

$$E_i^{(0)}:=\{\pm e_j\in {\mathbb{R}}^{n_i-1}:j=1,2,\dots ,n_i-1\},$$

$$\max \{\Vert A{x}^{(k)}-b\Vert ,\underset{i=1}{\overset{p}{\max }}\Vert {\bar{x}}^{i,(k)}\Vert -x_1^{i,(k)}\}\le 10^{-4}.$$

Full text

††journal: Numerical Algorithms

1 Introduction

In this paper, we consider the following second-order cone programming problem (SOCP):

[TABLE]

where $A\in\mathbb{R}^{m\times n},b\in\mathbb{R}^{m}$, and $c\in\mathbb{R}^{n}$ are a given matrix and vectors, and $\mathcal{K}$ denotes a Cartesian product of second-order cones (SOCs), i.e., $\mathcal{K}=\mathcal{K}^{n_{1}}\times\mathcal{K}^{n_{2}}\times\dots\times\mathcal{K}^{n_{p}}$ with $\mathcal{K}^{l}$ being an $l$-dimensional SOC, namely,

[TABLE]

If the SOCs in $\mathcal{K}$ are all one-dimensional, then the SOCP problem (1) reduces to the linear programming problem (LP) of the standard form:

[TABLE]

SOCP (1) is a very important optimization model, as it has many practical applications in fields such as robust optimization, antenna array problems, and beam forming problems lobo1998applications . To solve the SOCP, many researchers have developed algorithms exploiting the geometrical or algebraic structure of SOCs. For instance, we can find Newton-type methods such as primal-dual interior-point methods monteiro2000polynomial and non-interior continuous methods along with complementarity functions hayashi1 , Chubanov-type algorithms kitahara2018extension , and simplex-type algorithms hayashi2016simplex ; muramatsu2006pivoting . These algorithms were originally carried over from LP.

One popular extension from LP to SOCP is based on the Jordan algebra faraut1994analysis , whereby the two problems can be handled in the same algebraic framework. Another approach is based on the semi-infinite reformulation of the SOCP. By representing the SOCs as the intersection of an infinite number of half-spaces, the SOCP can be reformulated as the following linear semi-infinite programming problem (LSIP) with infinitely many linear inequality constraints:

[TABLE]

where $V_{i}:=\{v\in\mathbb{R}^{n_{i}-1}\mid\|v\|\leq 1\}$ if $n_{i}\geq 2$; otherwise, the corresponding constraint denotes $x^{i}\geq 0$ by convention, and $x^{i}\in\mathbb{R}^{n_{i}}$ denotes the $i$-th block of $x$ partitioned along the Cartesian structure of $\mathcal{K}$, i.e., $x=((x^{1})^{\top},(x^{2})^{\top},\dots,(x^{p})^{\top})^{\top}\in\prod_{i=1}^{p}\mathbb{R}^{n_{i}}$. Hayashi et al. hayashi2016simplex tailored the dual-simplex method for LP to the dual problem of SOCP (1) via the semi-infinite representation. For an overview of semi-infinite programming problems, we refer readers to survey articles sip1 ; sip2 .

The purpose of this paper is to extend the LP-Newton method for LP in the standard form (3) to SOCP (1). Algorithms for solving LP include the simplex method, ellipsoid method, and interior-point method. Although the ellipsoid and interior-point methods are polynomial-time algorithms, the existence of a strongly polynomial-time algorithm for solving LPs remains an open problem. In an attempt to devise a strongly polynomial-time algorithm for LPs, Fujishige et al. FHYZ09 proposed the LP-Newton method for box-constrained LPs, which have a box constraint $l\leq x\leq u$ instead of the nonnegativity constraint in LP (3). Kitahara et al. KMS13 extended this to LPs in the standard form (3). This algorithm repeats the projection of the current point onto a polytope arising from the feasible region and the computation of a supporting hyperplane and line. Numerical results in FHYZ09 suggest that relatively few iterations of the LP-Newton method are required, and hence the algorithm is considered promising.

Recently, Silvestri and Reinelt SR17 developed an LP-Newton method for SOCP. To the best of our knowledge, this is the first extension of the LP-Newton method to SOCP. In SR17 , the authors considered SOCP (1) with $x\in\mathcal{K}$ replaced by a box-like constraint $l\preceq x\preceq u$, which denotes $x-l,u-x\in\mathcal{K}$. Their algorithm computes a projection onto a conic zonotope at each iteration, and they proposed a Frank–Wolfe-based inner algorithm for this computation. Nevertheless, the computation of the projection still appears to be difficult. In fact, their numerical results show that the inner algorithm for obtaining the projection requires a number of iterations, although the outer loop is repeated relatively few times.

In this paper, we propose a different type of LP-Newton method for SOCP (1) based on the semi-infinite reformulation (4). In our approach, we construct a sequence of LPs by adaptively selecting finitely many constraints from the infinitely many constraints of LSIP (4). To produce an iteration point, we compute a projection onto a polytope arising from a polyhedral approximation of the SOCs, which can be realized by solving a convex quadratic programming problem (QP).

The remainder of this paper is organized as follows. In Section 2, we describe our proposed LP-Newton method for SOCP (1). In Section 3, we establish the global convergence of the proposed algorithm under the boundedness of the optimal set of SOCP (1). In Section 4, we propose a dual algorithm that generates a sequence in the dual space of SOCP (1). We also show its global convergence to an optimum of the dual problem of SOCP (1) under Slater’s constraint qualification. In Section 5, we report numerical results for the proposed method to investigate its validity and effectiveness.

2 Primal algorithm

In this section, we extend the LP-Newton method for LP (3) proposed by Kitahara et al. KMS13 to SOCP (1). For simplicity, we use the following notation:

[TABLE]

and for some $E_{i}\subseteq V_{i}\ (i=1,2,\dots,p)$,

[TABLE]

Moreover, we often denote $(1,v^{\top})^{\top}$ by $(1;v)$ for any vector $v$.

In the proposed algorithm, we construct a sequence of outer polyhedral approximations of the SOCs. By applying the LP-Newton method to the resulting LP, we update the polyhedral approximation of the SOCs. As a result, the algorithm generates a sequence $\{\mathcal{K}_{E^{(k)}}\}$ of adaptive outer approximations of the SOCs and a sequence $\{x^{(k)}\}$ of approximate optimal solutions to SOCP (1). We name the proposed algorithm the adaptive LP-Newton (ALPN) method for SOCP (1) and formally describe it as Algorithm 1.

In the computation of $\bar{w}^{(k)}$ and $x^{(k)}$ in Algorithm 1, we may solve the following QP:

[TABLE]

use an optimal solution as $x^{(k)}$, and set $\bar{w}^{(k)}=\bar{A}x^{(k)}$. If we solve QP (8) using the active set method, we can set $(\bar{w}^{(k)},x^{(k)})$ as an initial point of the $(k-1)$-th iteration. Despite the existence of a warm-start technique, solving QPs is still computationally expensive. Hence, a more sophisticated subroutine may be required. The LP-Newton method FHYZ09 for a box-constrained LP employs Wolfe’s algorithm Wol76 to find the nearest point in a zonotope to a given point, and the LP-Newton method KMS13 for the standard form LP (3) uses Wilhelmsen’s algorithm Wil76 to find the nearest point in a polyhedral cone to a given point. The subroutines are conjectured to be polynomial-time algorithms, and thus the LP-Newton methods for LPs have the potential to be strongly polynomial-time algorithms. Although these subroutines are powerful, it may be difficult to use them in Algorithm 1. In these subroutines, extreme directions or points of the zonotope or polyhedral cone are explicitly required. In our case, unfortunately, we do not have such explicit formulas.

Note that we can compute $\gamma^{(k+1)}$ in Algorithm 1 by

[TABLE]

In addition, it is easy to compute $v^{i,(k)}$ in Algorithm 1. In fact, an optimal solution to Problem (7) can be written in the following closed form:

[TABLE]

if $\bar{x}^{i,(k)}\neq 0$, where $\bar{x}^{i,(k)}\in\mathbb{R}^{n_{i}-1}$ denotes the subvector of $x^{i,(k)}$ without the first element, that is, $x^{i,(k)}=(x_{1}^{i,(k)},\bar{x}^{i,(k)})\in\mathbb{R}\times\mathbb{R}^{n_{i}-1}$.

3 Convergence analysis

In this section, we prove that a generated sequence converges globally to an optimum of SOCP (1). To this end, we make the following assumption:

Assumption 1

The optimal solution set $\mathcal{S}_{\mathrm{opt}}^{\mathrm{P}}$ of SOCP (1) is nonempty and compact.

Remark 1

Assumption 1 holds if the dual problem of (1):

[TABLE]

has an optimum and strictly feasible solution, i.e., there exists some $\bar{y}\in\mathbb{R}^{m}$ such that $A^{\top}\bar{y}-c\in\operatorname*{int}\mathcal{K}$.

We first state the following technical lemmas.

Lemma 1

Let $\{s^{(k)}\}$ be a sequence of nonnegative scalars and $\{z^{(k)}\}$ be defined by $z^{(k)}:=s^{(k)}x^{(k)}$ for each $k\geq 0$. If there exists an accumulation point of $\{z^{(k)}\}$, then it belongs to $\mathcal{K}$.

Proof

Let us express $z^{(k)}$ as $z^{(k)}:=(z^{1,(k)},z^{2,(k)},\dots,z^{p,(k)})=s^{(k)}(x^{1,(k)},x^{2,(k)},\dots,x^{p,(k)})$ and let $z^{\ast}:=(z^{1,\ast},z^{2,\ast},\dots,z^{p,\ast})$ be an arbitrarily chosen accumulation point of $\{z^{(k)}\}$. As $V^{i}$ is compact for every $i=1,2,\dots,p$, $\{v^{(k)}\}\subseteq V=\prod_{i=1}^{p}V_{i}$ has at least one accumulation point in $V$. Denote this point by $v^{\ast}$ and express it as $(v^{1,\ast},v^{2,\ast},\dots,v^{p,\ast})\in\prod_{i=1}^{p}V_{i}$. Taking an appropriate subsequence $\{v^{(k)}\}_{k\in S}$, we can assume that $(v^{(k)},z^{(k)})$ converges to $(v^{\ast},z^{\ast})$ as $k$ tends to $\infty$ in $S$.

Note that $v^{i,(k)}\in\operatorname*{argmin}_{v^{i}\in V_{i}}(1,(v^{i})^{\top})z^{i,(k)}$ holds because $v^{i,(k)}\in\operatorname*{argmin}_{v^{i}\in V_{i}}(1,(v^{i})^{\top})x^{i,(k)}$ and $s^{(k)}\geq 0$. Here, by letting $k\to\infty$ in $S$, we obtain $v^{i,\ast}\in\operatorname*{argmin}_{v^{i}\in V_{i}}(1,(v^{i})^{\top})z^{i,\ast}$. Thus, to show $z^{\ast}\in\mathcal{K}$, it suffices to prove that $(1,(v^{i,\ast})^{\top})z^{i,\ast}\geq 0$ for each $i$. To this end, let us fix $i$ and prove $(1,(v^{i,(k)})^{\top})z^{i,\ast}\geq 0$ for any $k\in S$. Choosing some arbitrary $\hat{k}\in S$, it follows that $(1,(v^{i,(\hat{k})})^{\top})z^{i,(k)}\geq 0$ for any $k>\hat{k}$ in $S$, because $z^{i,(k)}=s^{(k)}x^{i,(k)}\in\mathcal{K}_{E^{(k)}}$ and $v^{i,(\hat{k})}\in E_{i}^{(k)}$ for any $k>\hat{k}$. Then, by letting $k$ tend to $\infty$ in $S$, we obtain $(1,(v^{i,(\hat{k})})^{\top})z^{i,\ast}\geq 0$. As $\hat{k}$ was arbitrarily chosen from $S$, we conclude that $(1,(v^{i,(k)})^{\top})z^{i,\ast}\geq 0\ (k\in S)$ holds. Finally, by forcing $k\in S\to\infty$, we conclude that $(1,(v^{i,\ast})^{\top})z^{i,\ast}\geq 0$ for any $i$. Therefore, $z^{\ast}\in\mathcal{K}$. ∎

Lemma 2

Let $\theta^{\ast}$ be the optimal value of SOCP (1). If the algorithm does not stop at the $k$-th iteration, we have

[TABLE]

Proof

Let $\theta^{(k)}\in\mathbb{R}$ be the optimal value of LSIP (4) with $V_{i}$ replaced by $E_{i}^{(k)}$ for $i=1,2,\dots,p$, which is a relaxation problem for SOCP (1). Therefore, $\theta^{\ast}\leq\theta^{(k)}$ holds. In a similar manner to (KMS13, , Lemma 3.1), it can be verified that

[TABLE]

Hence, we have the desired result. ∎

Lemma 3

$\lim_{k\to\infty}b^{(k)}=b$* holds.*

Proof

To show the desired result, we prove $\lim_{k\to\infty}\|b^{(k)}-b\|=0$. Note that $\{\gamma^{(k)}\}$ and $\{\zeta^{(k)}\}$ converge to the same point by Lemma 2. Thus, $|\gamma^{(k)}-\gamma^{(k+1)}|\to 0$ and $|\gamma^{(k+1)}-\zeta^{(k)}|\to 0$ as $k\to\infty$. Moreover, as $\bar{w}^{(k)}$ is the projection of $w^{(k)}$ onto the supporting hyperplane $H^{(k)}$ and ${w}^{(k+1)}\in H^{(k)}$, we have

[TABLE]

From these facts, it follows that $\lim_{k\to\infty}\|b^{(k)}-b\|=0$. ∎

Proposition 1

If Assumption 1 holds, then the generated sequence $\{x^{(k)}\}$ is bounded.

Proof

Denote the feasible domain of SOCP (1) by $\mathcal{F}$. To show the boundedness of $\{x^{(k)}\}$, we assume to the contrary for a contradiction. Thus, there exists some subsequence $\{x^{(k)}\}_{k\in S}\subseteq\{x^{(k)}\}$ such that $\lim_{k\in S\to\infty}\|x^{(k)}\|=\infty$ and $\|x^{(k)}\|\neq 0$ for any $k\in S$. Then, we have

[TABLE]

where the second relation is derived from the fact that $x^{(k)}\in\mathcal{K}_{E^{(k)}}$ and $\mathcal{K}_{E^{(k)}}$ is a cone. Letting $k$ tend to $\infty$ in the above and choosing an arbitrary accumulation point of $\{x^{(k)}/\|x^{(k)}\|\}$, denoted by $d^{\ast}$, implies that

[TABLE]

where the first relation follows from the boundedness of $\{b^{(k)}\}$ implied by Lemma 3 and the second one follows from Lemma 1 with $z^{(k)}=x^{(k)}/\|x^{(k)}\|$ and $s^{(k)}=1/\|x^{(k)}\|$. Choose $\bar{z}\in\mathcal{S}_{\mathrm{opt}}^{\mathrm{P}}$ arbitrarily and define $\Omega:=\{\bar{z}+sd^{\ast}:s\geq 0\}$. We then deduce that $\Omega\subseteq\mathcal{S}_{\mathrm{opt}}^{\mathrm{P}}$ from Equation (16), because

[TABLE]

for any $s\geq 0$, where the second statement follows from the facts that $\bar{z}\in\mathcal{K}$, $sd^{\ast}\in\mathcal{K}$, and $\mathcal{K}$ is a convex cone. Note that $\Omega$ is unbounded because $\|d^{\ast}\|=1$, which implies the unboundedness of $\mathcal{S}_{\mathrm{opt}}^{\mathrm{P}}$. However, this contradicts Assumption 1. As a consequence, $\{x^{(k)}\}$ is bounded. ∎

Theorem 3.1

If Assumption 1 holds, any accumulation point of $\{x^{(k)}\}$ is an optimum of SOCP (1).

Proof

From Proposition 1, $\{x^{(k)}\}$ is bounded and has an accumulation point. Choose an arbitrary accumulation point and denote it by $x^{\ast}$. Without loss of generality, we may assume that $\lim_{k\to\infty}x^{(k)}=x^{\ast}$. Now, let us recall that $Ax^{(k)}=b^{(k)}$ and $x^{(k)}\in\mathcal{K}_{E^{(k)}}$ hold for any $k$. Together with Lemmas 1 and 3, this implies that $Ax^{\ast}=b$ and $x^{\ast}\in\mathcal{K}$, that is, $x^{\ast}$ is feasible for the SOCP. Hence, $c^{\top}x^{\ast}\leq\theta^{\ast}$ follows, where $\theta^{\ast}$ denotes the optimal value of the SOCP. However, by Lemma 2, it holds that $\theta^{\ast}\leq\zeta^{(k)}=c^{\top}x^{(k)}$ for any $k$, and by taking the limit therein, we obtain $\theta^{\ast}\leq c^{\top}x^{\ast}$. Therefore, we have $c^{\top}x^{\ast}=\theta^{\ast}$. Thus, we conclude that $x^{\ast}$ is optimal for the SOCP. ∎

Remark 2

According to (KMS13, , Theorem 3.1), for the case with $\mathcal{K}=\mathbb{R}_{+}^{n}$, the number of iterations of the algorithm is, at most, the number of faces of the cone $\bar{A}\mathcal{K}$.

4 Dual algorithm

4.1 Description of the algorithm

In Section 3, we proposed the ALPN method for solving SOCP (1). In this section, we consider a dual algorithm for the ALPN method, which solves the dual problem of SOCP (1) in dual variables $y\in\mathbb{R}^{m}$:

[TABLE]

In the dual algorithm, the following property plays a crucial role.

Proposition 2

Let $x^{\ast}$ be an optimum of SOCP (1) and $H^{\ast}$ be a supporting hyperplane of $\bar{A}\mathcal{K}$ at $\bar{A}x^{\ast}\in\bar{A}\mathcal{K}$. Suppose that $(1,-({y}^{\ast})^{\top})^{\top}$ is a normal vector to $H^{\ast}$. Then, together with $x^{\ast}$, ${y}^{\ast}$ and $\eta:=A^{\top}{y}^{\ast}-c$ satisfy the Karush–Kuhn–Tucker (KKT) conditions of SOCP (1):

[TABLE]

In particular, $y^{\ast}$ is an optimum of the dual SOCP (18).

Proof

As $H^{\ast}$ is a supporting hyperplane of $\bar{A}\mathcal{K}$ at $\bar{A}x^{\ast}=(c^{\top}x^{\ast},(Ax^{\ast})^{\top})^{\top}$ and $x^{\ast}$ solves SOCP (1), we have

[TABLE]

Hence, it holds that

[TABLE]

By the KKT conditions from (20), there exists some $\eta\in\mathbb{R}^{n}$ such that

[TABLE]

which, together with $Ax^{\ast}=b$, implies Equation (19). The optimality of $y^{\ast}$ for SOCP (18) is obvious. ∎

Our dual algorithm is described in Algorithm 2, where $\{x^{(k)}\}$ and $\{\gamma^{(k)}\}$ represent sequences generated by the ALPN method.

4.2 Convergence analysis

In addition to Assumption 1, we make the following assumption:

Assumption 2

Slater’s constraint qualification holds for SOCP (1), i.e., there exists some $z\in\mathbb{R}^{n}$ such that $z\in\operatorname*{int}\mathcal{K}$ and $Az=b$, and the matrix $A$ is of full row rank.

Under Assumptions 1 and 2, it is guaranteed that the optimal set of SOCP (18) is nonempty and compact.

Theorem 4.1

Under Assumptions 1 and 2, the generated sequence $\{y^{(k)}\}$ is bounded and any accumulation point of $\{y^{(k)}\}$ solves SOCP (18).

Proof

We first show the former claim. To construct a contradiction, suppose that $\{y^{(k)}\}$ is unbounded, and hence there exists some subsequence $\{y^{(k)}\}_{k\in S}$ such that $\|y^{(k)}\|\to\infty$ and $y^{(k)}\neq 0$ for any $k\in S$. Note that $H^{(k)}$ is a supporting hyperplane of $\bar{A}\mathcal{K}_{E^{(k)}}$ at $\bar{A}x^{(k)}$ that has the normal vector $(1,-(y^{(k)})^{\top})^{\top}$. Then, by the construction of $x^{(k)}$, we find that

[TABLE]

and thus

[TABLE]

Under the KKT conditions and the definition of $x^{(k)}$, we have

[TABLE]

where $\mathcal{K}_{E^{(k)}}^{\ast}$ denotes the dual cone of $\mathcal{K}_{E^{(k)}}$. As $\mathcal{K}_{E^{(k)}}^{\ast}\subseteq\mathcal{K}^{\ast}=\mathcal{K}$ follows from $\mathcal{K}\subseteq\mathcal{K}_{E^{(k)}}$, we find that $-c+A^{\top}y^{(k)}\in\mathcal{K}$. Divide $-c+A^{\top}y^{(k)}\in\mathcal{K}$ and $(-c+A^{\top}y^{(k)})^{\top}x^{(k)}=0$ by $\|y^{(k)}\|$ and let $k\in S\to\infty$ in Equation (25). Choose an accumulation point of $\{y^{(k)}/\|y^{(k)}\|\}$ and denote it by $d^{\ast}$. Let $x^{\ast}$ be an accumulation point of $\{x^{(k)}\}$ (recall Theorem 3.1). Without loss of generality, we may assume that $\lim_{k\in S\to\infty}(x^{(k)},y^{(k)}/\|y^{(k)}\|)=(x^{\ast},d^{\ast})$. Then, noting that $(-c+A^{\top}y^{(k)})/\|y^{(k)}\|\in\mathcal{K}$ for any $k\in S$ and $\lim_{k\to\infty}b^{(k)}=b$ by Lemma 3, it holds that

[TABLE]

Here, let $\bar{y}$ be an arbitrary optimum of SOCP (18). Then, the set $\Omega:=\{\bar{y}+sd^{\ast}:s\geq 0\}$ is contained by the optimal solution set of SOCP (18), denoted by $\mathcal{S}_{\mathrm{opt}}^{\mathrm{D}}$. This is shown as follows. Fix an arbitrary value of $s\geq 0$. Using the first relation in (26) and $-c+A^{\top}\bar{y}\in\mathcal{K}$, we have $-c+A^{\top}(\bar{y}+sd^{\ast})\in\mathcal{K}$, and so $\bar{y}+sd^{\ast}$ is feasible for SOCP (18). Moreover, it can be deduced from (26) that

[TABLE]

which indicates that the optimal value of SOCP (18) is also attained at $\bar{y}+sd^{\ast}$. Therefore, $\Omega\subseteq\mathcal{S}_{\mathrm{opt}}^{\mathrm{D}}$. Note that $\Omega$ is unbounded because $\|d^{\ast}\|\neq 0$, which implies the unboundedness of $\mathcal{S}_{\mathrm{opt}}^{\mathrm{D}}$. However, this contradicts the boundedness of $\mathcal{S}_{\mathrm{opt}}^{\mathrm{D}}$ derived from Assumptions 1 and 2. Hence, $\{y^{(k)}\}$ is bounded.

The second part of the claim is easy to prove by taking the limit in Equation (25) with the first relation replaced by $-c+A^{\top}y^{(k)}\in\mathcal{K}$. ∎

5 Numerical Results

We conducted numerical experiments to verify the performance of our proposed algorithm. We implemented the ALPN method with MATLAB R2018a (9.4.0.813654) on a workstation running CentOS release 6.10 with eight Intel Xeon CPUs (E3-1276 v3 3.60 GHz) and 32 GB RAM.

We used an initial polyhedral approximation of the $i$-th block of $\mathcal{K}$ with $n_{i}\geq 2$ given by

[TABLE]

where $e_{j}$ denotes the $j$-th column of the identity matrix. Note that $E_{i}^{(0)}$ defined above exactly represents $\mathcal{K}_{i}$ if $n_{i}=1,2$. In the projection step, we solved Problem (8) using the MATLAB function lsqlin. We stopped the algorithm when an approximate primal optimal solution was found, namely, a primal solution $x^{(k)}$ at the $k$-th iteration satisfies

[TABLE]

We randomly generated the following instances of SOCP (1). First, we set $m$, $p$, and $(n_{1},n_{2},\dots,n_{p})$ and randomly generated each element of $A$ from the standard Gaussian distribution. Next, we set $b=A\tilde{x}$ and $c=A^{\top}e-\tilde{s}$, where $e$ is the vector whose elements are all ones and

[TABLE]

Note that the two points $\tilde{x}$ and $\tilde{s}$ are interior feasible solutions of the primal and dual problems, respectively.

5.1 Performance of the Adaptive LP-Newton method

For the ALPN method, Table 1 presents the average runtime, number of iterations, and number of hyperplanes in the initial and final approximations of the SOC over ten executions. From this table, we can make the following observations:

• •

When $\mathcal{K}$ is polyhedral-like, i.e., $p\approx n$ and $n_{i}\approx 1$ for all $i=1,2,\dots,p$, the ALPN method works well. The algorithm gives a good polyhedral approximation of $\mathcal{K}$ with a small number of hyperplanes. As a result, there are few iterations and the computation time is short.

• •

When $\mathcal{K}$ is medium-dimensional, i.e., $1\ll p\ll n$ and $1\ll n_{i}\ll n$ for all $i=1,2,\dots,p$, the ALPN method becomes slow, although it gets better when $\mathcal{K}$ is high-dimensional, i.e., $p\approx 1$ and $n_{i}\approx n$ for all $i=1,2,\dots,p$. The medium-dimensional $\mathcal{K}$ requires many hyperplanes to obtain a good polyhedral approximation.

• •

The total dimension $n$ of the variables seems to be positively correlated with the runtime, although the runtime of the original LP-Newton method for LP is almost independent of $n$ FHYZ09 . This difference arises from the solution methods of the minimum norm point step. In our implementation, we solve Problem (8) using the MATLAB function lsqlin, for which the computation time depends on $n$.

• •

Surprisingly, the number $m$ of linear constraints is negatively correlated with the runtime. This might be because the dimension of the feasible region is low for large values of $m$, and this region can then be approximated by a small number of hyperplanes.

5.2 Comparison with the primal-dual interior-point method

We also compared our proposed ALPN method with the primal-dual interior-point method. In this experiment, we solved the randomly generated instances using our implementation of ALPN and SDPT3 TTT03 , which is a MATLAB implementation of the primal-dual interior-point method. Basically, SDPT3 was found to be faster than the ALPN method. However, the computation time of SDPT3 increases with $m$, whereas that of ALPN decreases as $m$ and $p$ increase. For instances with large values of $m$ and $p$, ALPN outperformed SDPT3. The results are presented in Table 2, which shows the average runtime of the ALPN and SDPT3 methods over ten runs.

6 Conclusions

In this paper, we have developed an LP-Newton method for SOCP through a transformation into LSIP with an infinite number of linear inequality constraints. The proposed ALPN algorithm produces a sequence by sequentially projecting the current point onto a polyhedral cone arising from finitely many linear inequality constraints chosen from the constraints of the LSIP. We also proposed a dual algorithm for the ALPN method for solving the dual of the SOCP. Under some mild assumptions, we proved that arbitrary accumulation points of the sequences generated by the two proposed algorithms are optima of the SOCP and its dual. Finally, we conducted some numerical experiments and compared the performance of our algorithms with that of the primal-dual interior point method. Future work will consider the extension of the ALPN method to semi-definite programming problems or symmetric cone programming problems.