Signomial and Polynomial Optimization via Relative Entropy and Partial Dualization

TL;DR

This paper introduces a generalized SAGE relaxation method for bounding constrained signomial and polynomial optimization problems, leveraging relative entropy and convex duality for improved solution insights.

Contribution

It extends SAGE relaxations to a broader class of problems using relative entropy, offering better transparency and a new perspective-based solution recovery method.

Findings

Effective bounds on complex optimization problems

Preserves sparsity in solutions

Demonstrated utility with examples and software

Abstract

We describe a generalization of the Sums-of-AM/GM Exponential (SAGE) relaxation methodology for obtaining bounds on constrained signomial and polynomial optimization problems. Our approach leverages the fact that relative entropy based SAGE certificates conveniently and transparently blend with convex duality, in a manner that Sums-of-Squares certificates do not. This more general approach not only retains key properties of ordinary SAGE relaxations (e.g. sparsity preservation), but also inspires a novel perspective-based method of solution recovery. We illustrate the utility of our methodology with a range of examples from the global optimization literature, along with a publicly available software package.

Tables

Table 1. Table 1: SAGE bounds for Example 1, with solver runtime for Machines 𝗪 𝗪 \bm{\mathsf{W}} and 𝗟 𝗟 \bm{\mathsf{L}} . A level- 3 3 3 bound certifies the level-0 solution as optimal, within relative error 10 − 8 superscript 10 8 10^{-8} .

level	SAGE bound	$𝗪$ time (s)	$𝗟$ time (s)
0	-147.85713	0.03	0.01
1	-147.67225	0.05	0.02
2	-147.66680	0.08	0.08
3	-147.66666	0.19	0.26

Table 2. Table 2: Columns n 𝑛 n and k 𝑘 k give number of variables and number of inequality constraints for the indicated problem. “Solution quality” is “same” (resp. “improved”) if Algorithm 1 L returned a feasible solution with objective equal to (resp. less than) that proposed in [ RM1978 ] . Problems 9, 12, and 14 are discussed in Table 3 . We encountered solver failures for level- ( 1 , 1 , 0 ) 1 1 0 (1,1,0) relaxations of problems 13, 17, and 18.

Num. in [RM1978]	$n$	$k$	solution quality	optimal?
9	2	1	same	unknown
10	3	1	improved	yes
11	4	2	same	yes
12	8	4	same	unknown
13	8	6	no solution	no
14	10	7	same	unknown
15	10	7	same	yes
16	10	7	same	yes
17	11	8	no solution	no
18	13	9	no solution	no

Table 3. Table 3: Problems for which we did not certify optimality, but nevertheless recovered best-known solutions by using SAGE relaxations. Note that Algorithm 1 returned strictly-feasible solutions in each of these cases. In the next section we present examples where Algorithm 1 does not return feasible solutions, and so solution refinement (i.e. Algorithm 1 L) becomes more important.

	SAGE relaxation		Algorithm 1		Algorithm 1L
Num. in [RM1978]	$(p,q,\ell )$	bound	$f\left(𝒙\right)$	$\min g\left(𝒙\right)$	$f\left(𝒙\right)$	$\min g\left(𝒙\right)$
9	(3,3,1)	11.7	12.500	0.00438	11.9600	2.00E-10
12	(0,2,1)	-6.4	-5.7677	0.00034	-6.0482	-5.00E-09
14	(0,4,0)	0.7	2.5798	0.01541	1.14396	-8.00E-09

Table 4. Table 4: Columns n 𝑛 n , k 1 subscript 𝑘 1 k_{1} , and k 2 subscript 𝑘 2 k_{2} specify the number of variables, inequality constraints, and equality constraints in the indicated problem. The last three columns specify the objective value and constraint violation of a solution obtained by running Algorithm 1 L on the output of a dual SAGE relaxation, as well as a note on whether the objective matched a SAGE bound. Problems with “unknown” optimality status are described in Table 5 .

source	num.	$n$	$k_1$	$k_2$	objective	infeasibility	optimal?
[HSC2014]	1	4	10	0	0.7650822	0	yes
-	2	2	5	0	11.964337	0	yes
-	3	3	7	0	-147.66666	0	yes
-	5	5	16	0	10122.493	4.00E-13	unknown
-	7	3	6	0	-10.363636	2.00E-15	yes
-	8	15	37	6	156.21963	4.00E-14	yes
[Xu2014]	4	2	1	1	1.3934649	2.00E-10	yes
-	5	6	9	4	-0.3888114	5.00E-17	unknown
-	6	2	4	2	1.1771243	4.00E-12	yes
-	7	6	20	3	10252.790	8.00E-14	unknown

Table 5. Table 5: Signomial programs for which we did not certify optimality, but nevertheless recovered best-known solutions by using SAGE relaxations. Columns ϵ ineq subscript italic-ϵ ineq \epsilon_{\text{ineq}} and ϵ eq subscript italic-ϵ eq \epsilon_{\text{eq}} indicate the value of infeasibility tolerances when running Algorithm 1, prior to feeding the output of Algorithm 1 to COBYLA as part of Algorithm 1L. The last two columns list the objective function value and constraint violations for the output of Algorithm 1L. [ Xu2014 ] -7 reports a solution with smaller objective value, however that solution violates an equality constraint with forward error in excess of 0.11.

source-num.	$(p,q,\ell )$	bound	${ϵ}_{\text{ineq}}$	${ϵ}_{\text{eq}}$	objective	infeasibility
[HSC2014]-5	(0,1,0)	$9171.00$	1.00E-08	0	$10122.493$	4.00E-13
[Xu2014]-5	(2,2,0)	-0.390	1.00E-08	1	-0.3888114	5.00E-17
[Xu2014]-7	(0,1,0)	$9397.8$	1.00E-08	1	$10252.790$	8.00E-14

Table 6. Table 6: Results for SAGE on unconstrained and box-constrained polynomial minimization problems. Column “ d 𝑑 d ” indicates the degree of the polynomial to be minimized. The Rosenbrock example allows for different numbers of variables, though results from [ SAGE2 ] show SAGE is tight for any number of variables. The Beale, Colville, and Goldstein-Price polynomials proved very difficult for optimization via SAGE certificates.

source	name	$n$	$d$	minimum	SAGE solved
[VLSE]	Rosenbrock	variable	4	0	yes
-	6-hump camel	2	6	-1.0316	yes
-	3-hump camel	2	6	0	yes
-	Beale	2	8	0	no
-	Colville	4	4	0	no
-	Goldstein-Price	2	8	3	no
[RN2008]	L.V. 4	4	4	-20.8	yes
-	Cap 4	4	4	-3.117690	yes
-	Hun 5	5	7	-1436.515	no
-	Cyc 5	5	4	-3	yes
-	C.D. 6	6	2	-270397.4	no
-	But 6	6	3	-1.4393	yes
-	Heart 8	8	4	-1.367754	yes
-	Viras 8	8	2	-29	yes

Table 7. Table 7: Generic polynomial optimization problems, over the nonnegative orthant. Problems can be found in both [ LTY2017 ] and [ WLT2018 ] ; names “ P n 𝑛 n _ d 𝑑 d ” indicate the number of variables n 𝑛 n and degree d 𝑑 d of the given problem. Column k 𝑘 k gives the number of inequality constraints, excluding constraints 𝒙 ≥ 𝟎 𝒙 0 \bm{x}\geq\bm{0} , as well as those which trivially follow from 𝒙 ≥ 𝟎 𝒙 0 \bm{x}\geq\bm{0} . SAGE solved all problems listed here, at the indicated level of the hierarchy, and with the indicated solver runtimes (in seconds).

name	$k$	minimum	$(p,q,\ell )$	$𝗪$ time (s)	$𝗟$ time (s)
P4_4	8	-0.033538	(1,1,0)	0.47	0.7
P4_6	7	-0.060693	(1,1,1)	289	292
P4_8	7	-0.085813	(2,1,0)	396	460
P6_4	8	-0.576959	(1,1,0)	3.45	4.1
P6_6	8	-0.412878	(1,1,0)	3.04	4.37
P6_8	8	-0.409020	(1,1,0)	3.25	3.83
P8_4	8	-0.436026	(1,1,0)	7.18	7.25
P8_6	8	-0.412878	(1,1,0)	8.67	8.21

Table 8. Table 8: Comparison of Algorithm 2 and Algorithm 2 L for solution recovery for eight nonconvex polynomial optimization problems in the literature (ref. [ LTY2017 , WLT2018 ] ). Both algorithms were initialized with solutions to a level-(1,1,0) conditional SAGE relaxation, and Algorithm 2 L always recovers an optimal solution. It is especially notable that Algorithm 2 L recovers optimal solutions for problems P4_6 and P4_8 , since level-(1,1,0) relaxations do not produce tight bounds for these problems.

	Algorithm 2		Algorithm 2L
name	$f\left(𝒙\right)$	$\min g\left(𝒙\right)$	$f\left(𝒙\right)$	$\min g\left(𝒙\right)$
P4_4	-0.033386	0.00E-00	-0.033538	0.00E-00
P4_6	-0.057164	4.06E-02	-0.060693	-2.44E-14
P4_8	-0.066671	1.42E-01	-0.085813	-3.46E-14
P6_4	-0.570848	4.04E-08	-0.576959	-1.03E-13
P6_6	-0.412878	5.46E-09	-0.412878	-1.68E-13
P6_8	-0.409018	1.07E-07	-0.409020	-5.82E-14
P8_4	-0.436024	3.27E-08	-0.436026	-2.58E-13
P8_6	-0.412878	2.78E-43	-0.412878	-2.55E-12

Table 9. Table 9: Solver runtimes for level-(0,2,0) conditional SAGE relaxations, on Machine 𝗪 𝗪 \bm{\mathsf{W}} . Similar runtimes can be expected for Machine 𝗟 𝗟 \bm{\mathsf{L}} with n ∈ { 10 , 20 , 30 } 𝑛 10 20 30 n\in\{10,20,30\} . Solve times with Machine 𝗟 𝗟 \bm{\mathsf{L}} can take much longer for n ≥ 40 𝑛 40 n\geq 40 , since only a portion of the problem fits in RAM.

solve time (s)	$n=10$	$n=20$	$n=30$	$n=40$
mean	7.54E-01	6.50E-00	6.46E+01	4.59E+02
std dev.	8.74E-02	8.54E-01	1.38E+01	7.20E+01

Equations

$${\mathsf{C}}_{\mathsf{NNS}}(\mathbf{\alpha },X)\doteq \{\mathbf{c}:\mathrm{Sig}(\mathbf{\alpha },\mathbf{c})(\mathbf{x})\ge 0\text{ for all }\mathbf{x}\text{ in }X\}$$

$${\mathsf{C}}_{\mathsf{NNP}}(\mathbf{\alpha },X)\doteq \{\mathbf{c}:\mathrm{Poly}(\mathbf{\alpha },\mathbf{c})(\mathbf{x})\ge 0\text{ for all }\mathbf{x}\text{ in }X\}.$$

$$D(\mathbf{u},\mathbf{v})=\sum _{i=1}^m{u}_i\log (u_i/v_i).$$

$$(f,g,\varphi {)}_X^{\star }=\inf \{f(\mathbf{x}):\mathbf{x}\text{ in }X\subset {\mathbb{R}}^n,g(\mathbf{x})\ge 0,\varphi (\mathbf{x})=0\}$$

$$(f,g{)}_X^{\star }=\inf \{f(\mathbf{x}):\mathbf{x}\text{ in }X\subset {\mathbb{R}}^n,g(\mathbf{x})\ge 0\}.$$

$$(f,g{)}_{{\mathbb{R}}^n}^{\mathrm{L}}=\underset{\mathbf{\lambda }\ge 0}{\sup }\underset{\mathbf{x}\in {\mathbb{R}}^n}{\inf }\mathcal{L}(\mathbf{x},\mathbf{\lambda }).$$

$$(f,g{)}_{{\mathbb{R}}^n}^{\mathrm{L}}=\sup \{\gamma :\mathbf{\lambda }\ge 0,\gamma \text{ in }\mathbb{R},\psi (\gamma ,\mathbf{\lambda })(\mathbf{x})\ge 0\text{ for all }\mathbf{x}\text{ in }{\mathbb{R}}^n\},$$

$${\mathsf{C}}_{\mathsf{AGE}}(\mathbf{\alpha },k)=\{\mathbf{c}:{\mathbf{c}}_{\setminus k}\ge 0\text{ and }\mathbf{c}\text{ belongs to }{\mathsf{C}}_{\mathsf{NNS}}(\mathbf{\alpha })\}.$$

$$\text{some}\mathbf{\nu }\text{ in }{\mathbb{R}}_{+}^{m-1}\text{has}[{\mathbf{\alpha }}_{\setminus i}-1{\mathbf{\alpha }}_i{]}^{⊺}\mathbf{\nu }=0\text{and}D(\mathbf{\nu },{\mathbf{c}}_{\setminus k})-{\mathbf{\nu }}^{⊺}1\le c_k.$$

$${\mathsf{C}}_{\mathsf{SAGE}}(\mathbf{\alpha })\doteq \left\{\mathbf{c}:\text{ there exist }{\mathbf{c}}^{(k)}\text{ in }{\mathsf{C}}_{\mathsf{AGE}}(\mathbf{\alpha },k)\text{ satisfying }\mathbf{c}=\sum _{k=1}^m{\mathbf{c}}^{(k)}\right\}$$

$${\mathsf{C}}_{\mathsf{AGE}}^{\mathsf{POLY}}(\mathbf{\alpha },k)=\{\mathbf{c}:$$

$${\mathbf{c}}_{\setminus k}\ge 0,c_i=0\text{ for all }i\ne k\text{ with }{\mathbf{\alpha }}_i\notin 2{\mathbb{N}}^{1\times n}\},$$

$${\mathsf{C}}_{\mathsf{SAGE}}^{\mathsf{POLY}}(\mathbf{\alpha })\doteq \sum _{k=1}^m{\mathsf{C}}_{\mathsf{AGE}}^{\mathsf{POLY}}(\mathbf{\alpha },k)\subset {\mathsf{C}}_{\mathsf{NNP}}(\mathbf{\alpha }).$$

$$\mathrm{SR}(\mathbf{\alpha },\mathbf{c})=\{\hat{\mathbf{c}}$$

$${\hat{c}}_i\le -|c_i|\text{ whenever }{\mathbf{\alpha }}_i\text{ is not in }2{\mathbb{N}}^{1\times n}\}.$$

$${\mathsf{C}}_{\mathsf{SAGE}}^{\mathsf{POLY}}(\mathbf{\alpha })=\{\mathbf{c}:$$

$$\mathrm{SOS}(n,2d)=\{p:p(\mathbf{x})=L_d^n(\mathbf{x}{)}^{⊺}\mathbf{M}{L}_d^n(\mathbf{x}),\mathbf{M}⪰0\}$$

$$\inf \{-x^2:-1\le x\le 1\}=-1.$$

$$\inf \{-\exp (2x):1\le \exp (x)\le 2\}=-4.$$

$$f_{{\mathbb{R}}^n}^{\star }\doteq \inf \{f(\mathbf{x}):\mathbf{x}\text{ in }{\mathbb{R}}^n\}=\sup \{\gamma :f(\mathbf{x})-\gamma \ge 0\text{ for all }\mathbf{x}\text{ in }{\mathbb{R}}^n\}.$$

$$f_{{\mathbb{R}}^n}^{(\ell )}=\sup \{\gamma :\gamma \text{ in }\mathbb{R},w^{\ell }(f-\gamma )\text{ is SAGE}\}\le f_{{\mathbb{R}}^n}^{\star }.$$

$$(f,g{)}_X^{\mathrm{d}}\doteq \sup \{\gamma :\mathbf{\lambda }\ge 0,\gamma \text{ in }\mathbb{R},\mathcal{L}(\mathbf{x},\mathbf{\lambda })-\gamma \ge 0\text{ for all }\mathbf{x}\text{ in }X\}.$$

$$(f,g{)}_{[a,b]}^{\mathrm{d}}=\sup \{\gamma :\gamma ,\mu \in \mathbb{R},f(x)-\mu g(x)-\gamma \ge 0\text{ for all }x\text{ in }[a,b]\}$$

$$(f,g{)}_{[a,b]}^{\mathrm{d}}=\sup \{\gamma :$$

$$s\in \mathrm{SOS}(1,2d),t\in \mathrm{SOS}(1,2(d-1))\}.$$

$$f_X^{\star }=\sup \{\gamma :\mathbf{c}-\gamma (1,0,\dots ,0)\in {\mathsf{C}}_{\mathsf{NNP}}(\mathbf{\alpha },X)\}.$$

$$f_X^{\star }=\inf \{{\mathbf{c}}^{⊺}\mathbf{v}:\mathbf{v}\in {\mathsf{C}}_{\mathsf{NNP}}(\mathbf{\alpha },X{)}^{†},v_1=1\}$$

$$({\mathbb{R}}^n{)}^{\mathbf{\alpha }}\doteq \{({\mathbf{x}}^{{\mathbf{\alpha }}_1},\dots ,{\mathbf{x}}^{{\mathbf{\alpha }}_m}):\mathbf{x}\in {\mathbb{R}}^n\},$$

$${\mathsf{C}}_{\mathsf{AGE}}(\mathbf{\alpha },k,X)=\{\mathbf{c}\in {\mathbb{R}}^m:{\mathbf{c}}_{\setminus k}\ge 0\text{ and }\mathrm{Sig}(\mathbf{\alpha },\mathbf{c})(\mathbf{x})\ge 0\text{ for all }\mathbf{x}\text{ in }X\}.$$

$${\mathsf{C}}_{\mathsf{SAGE}}(\mathbf{\alpha },X)\doteq \sum _{k=1}^m{\mathsf{C}}_{\mathsf{AGE}}(\mathbf{\alpha },k,X)$$

$$f_X^{\mathrm{SAGE}}\doteq \sup \{\gamma :\gamma \text{ in }\mathbb{R},\mathbf{c}-\gamma (1,0,\dots ,0)\text{ in }{\mathsf{C}}_{\mathsf{SAGE}}(\mathbf{\alpha },X)\}$$

$${\mathbf{c}}^{(i)}\in {\mathsf{C}}_{\mathsf{AGE}}(\mathbf{\alpha },i,X)\mathbf{c}=\sum _{i\in \mathcal{N}}{\mathbf{c}}^{(i)}\text{ and }{c}_j^{(i)}=0\text{ for all }j\ne i\text{ in }\mathcal{N}.$$

Full text

Keywords: global optimization, exponential cone programs, SAGE certificates, SOS certificates, signomial programming.

1 Introduction

A signomial is a function of the form $\bm{x}\mapsto\sum_{i=1}^{m}c_{i}\exp(\bm{\alpha}_{i}\cdot\bm{x})$ for real scalars $c_{i}$ and row vectors $\bm{\alpha}_{i}$ in $\mathbb{R}^{1\times n}$. Signomial optimization (often called signomial programming) concerns the minimization of a signomial, subject to signomial inequality and equality constraints. Signomial programming is a computationally challenging problem with applications in chemical engineering [Rountree1982], aeronautics [OCW2017], circuit design [jabr2007], and communications network optimization [Chiang2009]. Signomials are sometimes thought of as generalizations of polynomials over the positive orthant; by a change of variables $y_{i}=\exp x_{i}$ one arrives at “geometric form” signomials $\bm{y}\mapsto\sum_{i=1}^{m}c_{i}\prod_{j=1}^{n}y_{j}^{\alpha_{ij}}$. Despite this aesthetic similarity between polynomials and geometric-form signomials, we must bear in mind that signomials and polynomials have many significant differences. Where polynomials can be generated by a countably infinite basis, signomials require an uncountably infinite basis. Where polynomials are closed under composition, signomials are not. Where polynomials and exponential-form signomials are defined on all of $\mathbb{R}^{n}$ – geometric-form signomials are only defined on the positive orthant.

For many years these abstract differences between signomials and polynomials have coincided with algorithmic disparities. Contemporary methods for signomial programming use some combination of local linearization, penalty functions, sequential geometric programming, and branch-and-bound [Shen2004, WL2005, Shen2006, Qu2007, Shen2008, HSC2014, Xu2014] – ideas which precede the advent of modern convex optimization. By contrast, the field of polynomial optimization has been substantially influenced by semidefinite programming, specifically through Sums-of-Squares (SOS) certificates of polynomial nonnegativity [shor, ParriloPhD, Lasserre2001]. In recent work, Chandrasekaran and Shah proposed the Sums-of-AM/GM Exponential or “SAGE” certificates of signomial nonnegativity, which provided a new convex relaxation framework for signomial programs akin to SOS methods for polynomial optimization. [SAGE1]. Where SOS certificates make use of semidefinite programming, SAGE certificates use the convex relative entropy function. The authors of the present article further demonstrated that a natural modification to SAGE certificates leads to a tractable relative entropy representable sufficient condition for global polynomial nonnegativity [SAGE2].

This article is concerned with how proof systems for function nonnegativity can be used in the service of constrained optimization. The basic idea here is simple: for a function $f$, a set $X$, and a real number $\gamma$, we have $\inf\{f(\bm{x})\,:\,\bm{x}\in X\}\geq\gamma$ if and only if $f-\gamma$ is nonnegative over $X$. The trouble is that to leverage this fact, we require ways to extend certificates for global nonnegativity (such as SOS or SAGE certificates) to prove nonnegativity over $X\subsetneq\mathbb{R}^{n}$. For the polynomial case one usually performs this extension by appealing to representation theorems from real algebraic geometry. In the absence of such representation theorems, one typically relies on a dual problem obtained from the minimax inequality.

The primary contribution of this article is to show how SAGE certificates – by virtue of their roots in convex duality – provide a simple and powerful alternative method for describing functions which are nonnegative over proper subsets of $\mathbb{R}^{n}$. Our method can be used both independently from and in conjunction with the minimax inequality. The space of possibilities with our method is large, and it is far from obvious as to which variations of this methodology are most useful for given problem structures. To facilitate research in this regard, we provide a user-friendly software package which implements all functionality described in this article. We provide detailed worked examples in several places alongside conceptual development. A dedicated section on computational experiments is provided, and several avenues of possible future research are outlined in a discussion section.

1.1 Article outline and our contributions

This article makes both mathematical and methodological contributions to signomial and polynomial optimization. Section 2 speaks to key questions which help place our work in a broader context. These questions include (1) What are the sources of error in nonnegativity-based relaxations of constrained optimization problems, and how are they usually mitigated? (2) How exactly are the original SAGE cones formulated? (3) How can we understand partial dualization in the context of existing nonnegativity and moment relaxations?

Once these questions are answered, we introduce the concept of conditional SAGE certificates for signomial nonnegativity (Section 3). We prove a representation result for the cone of these nonnegativity certificates (Theorem 6), and develop a solution recovery algorithm by investigating the dual cone (Algorithm 1). Section 3.4 describes two “hierarchies” of SAGE-based convex relaxations for signomial programs: one which uses the minimax inequality, and one which is minimax-free. The authors know of no analog to the minimax-free hierarchy in the polynomial optimization literature, and believe the underlying idea of the minimax-free hierarchy is of independent theoretical interest.

Section 4 extends the idea of conditional SAGE certificates to polynomials. We discuss basic properties of the conditional SAGE polynomial cones before proving representation results (Theorems 9 and 10) which provide the basis for tractable relaxations of constrained polynomial optimization problems. Section 4.2 provides simple descriptions for dual conditional SAGE polynomial cones, and develops an efficient solution recovery algorithm based on these descriptions (Algorithm 2). Section 4.4 proposes reference hierarchies for polynomial optimization with SAGE certificates. Our minimax-free hierarchy has an interesting structure which reflects a link between SAGE signomials and SAGE polynomials, by way of the “signomial representatives” from [SAGE2].

Section 5 reports the effectiveness of our methodology on fifty-one problems appearing in the literature (sourced from [Yan1976, RM1978, RN2008, HSC2014, Xu2014, LTY2017, WLT2018, VLSE]), as well as randomly generated problems. A central component of our experiments is a desire to facilitate research both into theory underlying conditional SAGE relaxations, and the practice of using these relaxations in engineering design optimization. Towards this end, we provide the “sageopt” Python package.111https://rileyjmurray.github.io/sageopt/ Sageopt is a documented, tested, and convenient platform for constructing and solving SAGE relaxations, as well as analyzing the results thereof. We used sageopt for all experiments in this article.

1.2 Notation and preliminary definitions

Vectors and matrices always appear in boldface. The $i^{\text{th}}$ entry of a vector $\bm{v}$ is $v_{i}$, and the vector formed by deleting the $i^{\text{th}}$ entry of $\bm{v}$ is $\bm{v}_{\setminus i}$. A matrix $\bm{A}$ is built by stacking rows $\bm{a}_{i}\in\mathbb{R}^{1\times n}$, and $\bm{A}_{\setminus i}$ is the submatrix formed by deleting the $i^{\text{th}}$ row of $\bm{A}$. All logarithms in this article are base-$e$. Elementary functions from $\mathbb{R}$ to $\mathbb{R}$ are extended first to vectors in an elementwise fashion, and subsequently to sets in a pointwise fashion. For a convex cone $K\subset\mathbb{R}^{r}$, the dual cone is $K^{\dagger}\doteq\{\bm{y}\,:\,\bm{y}^{\intercal}\bm{x}\geq 0\text{ for all }\bm{x}\text{ in }\mathbb{R}^{r}\}$. For $A,B\subset\mathbb{R}^{n}$, $A\subset B$ and $A\subsetneq B$ denote non-strict and strict inclusion respectively. The operator “$\operatorname{cl}$” computes set-closure with respect to the standard topology.

For an $m\times n$ matrix $\bm{\alpha}$ and a vector $\bm{c}$ in $\mathbb{R}^{m}$, we write $f=\operatorname{Sig}(\bm{\alpha},\bm{c})$ to mean that $f$ takes values $f(\bm{x})=\sum_{i=1}^{m}c_{i}\exp(\bm{\alpha}_{i}\cdot\bm{x})$. When $\bm{\alpha}$ is a matrix of nonnegative integers, we write $f=\operatorname{Poly}(\bm{\alpha},\bm{c})$ to mean that $\bm{c}$ is the coefficient vector of $f$ with respect to the monomial basis $\bm{x}\mapsto\bm{x}^{\bm{\alpha}_{i}}\doteq\prod_{j=1}^{n}x_{j}^{\alpha_{ij}}$. Given a matrix $\bm{\alpha}$ and a set $X\subset\mathbb{R}^{n}$, one has the nonnegativity cones

[TABLE]

and

[TABLE]

We write $\mathsf{C}_{\mathsf{NNS}}(\bm{\alpha})$ and $\mathsf{C}_{\mathsf{NNP}}(\bm{\alpha})$ in reference to the above cones when $X=\mathbb{R}^{n}$. Except in special cases on $\bm{\alpha}$, it is computationally intractable to check membership in either $\mathsf{C}_{\mathsf{NNS}}(\bm{\alpha})$ or $\mathsf{C}_{\mathsf{NNP}}(\bm{\alpha})$ [Murty1987]. The inner-approximations of nonnegativity cones developed in this article make use of the relative entropy function; this is the convex function “$D$” with domain $\mathbb{R}^{m}_{+}\times\mathbb{R}^{m}_{+}$ taking values

[TABLE]

This article includes computational experiments with SAGE certificates and states solver runtimes for many of these examples. All of these examples rely on the MOSEK solver [mosek]. We use two different machines to provide a sense of when it may be practical to solve a SAGE relaxation with given computational resources. Machine $\bm{\mathsf{W}}$ is an HP Z820 workstation, with two 8-core 2.6GHz Intel Xeon E5-2670 processors and 256GB 1600MHz DDR3 RAM. Machine $\bm{\mathsf{L}}$ is a 2013 MacBook Pro, with a dual-core 2.4GHz Intel Core i5 processor and 8GB 1600MHz DDR3 RAM.

2 Background

In this article we study constrained nonconvex optimization problems of the form

[TABLE]

where $f$ is a function from $\mathbb{R}^{n}$ to $\mathbb{R}$, $g$ maps $\mathbb{R}^{n}$ to $\mathbb{R}^{k_{1}}$, and $\phi$ maps $\mathbb{R}^{n}$ to $\mathbb{R}^{k_{2}}$. Our primary goal is to produce lower bounds $(f,g,\phi)^{\mathrm{lb}}_{X}\leq(f,g,\phi)^{\star}_{X}$. In the event that $(f,g,\phi)^{\mathrm{lb}}_{X}=(f,g,\phi)^{\star}_{X}$, we are also interested in recovering optimal solutions to (1). For ease of exposition, this section focuses on problems of the form (1) with only inequality constraints– i.e. the problem of bounding

[TABLE]

In Section 2.1 we review the Lagrange dual relaxation of the above problem, both in minimax form and as a nonnegativity problem. Section 2.2 provides the minimum background on SAGE and SOS nonnegativity certificates needed develop the contributions of this article. In Section 2.3 we review standard techniques for strengthening nonnegativity-based relaxations of problems such as (1.1); this includes the use of redundant constraints, nonconstant Lagrange multipliers, and strengthening nonnegativity certificates via modulation. Section 2.4 concludes with discussion on partial dualization. Until Section 2.4, the set $X$ appearing in Problem 1.1 shall be the whole of $\mathbb{R}^{n}$.

2.1 Dual problems in nonconvex optimization

The simplest way to lower bound $(f,g)^{\star}_{\mathbb{R}^{n}}$ is via the Lagrange dual. For each coordinate function $g_{i}$ of $g$, we introduce a dual variable $\lambda_{i}\geq 0$ and consider the Lagrangian $\mathcal{L}(\bm{x},\bm{\lambda})=f(\bm{x})-\bm{\lambda}^{\intercal}g(\bm{x})$. The Lagrange dual problem is to compute

[TABLE]

By the minimax inequality, we can be certain that $(f,g)^{\mathrm{L}}_{\mathbb{R}^{n}}\leq(f,g)^{\star}_{\mathbb{R}^{n}}$.

There are many situations when the Lagrange dual problem is intractable. For signomial and polynomial optimization, one usually needs to compute yet another lower bound $(f,g)_{\mathbb{R}^{n}}^{\mathrm{d}}\leq(f,g)_{\mathbb{R}^{n}}^{\mathrm{L}}$. Contemporary approaches for computing such bounds begin by introducing a parameterized function $\psi(\gamma,\bm{\lambda})$ which takes values $\psi(\gamma,\bm{\lambda})(\bm{x})=\mathcal{L}(\bm{x},\bm{\lambda})-\gamma$. One reformulates the dual problem as

[TABLE]

and the constraint that “$\psi(\gamma,\bm{\lambda})$ defines a nonnegative function” is then tightened to “$\psi(\gamma,\bm{\lambda})$ satisfies a particular sufficient condition for nonnegativity.” The expectation is that the sufficient condition can be expressed by tractable convex constraints on variables $\gamma$ and $\bm{\lambda}$. For example, SOS certificates for polynomial nonnegativity can be expressed via linear matrix inequalities, and SAGE certificates for signomial and polynomial nonnegativity can be expressed with the relative entropy function.

2.2 SAGE and SOS nonnegativity certificates

In the development of the SAGE inner approximation for $\mathsf{C}_{\mathsf{NNS}}(\bm{\alpha})$, Chandrasekaran and Shah considered the structure where the coefficient vector $\bm{c}$ contained at most one negative entry $c_{k}$; if such a function was globally nonnegative, they called it an AM/GM Exponential, or an AGE function [SAGE1]. One thus defines the $k^{\text{th}}$* AGE cone*

[TABLE]

A key contribution of [SAGE1] was the use of convex duality to derive an efficient description of the AGE cones. The outcome of this derivation is that a vector $\bm{c}$ belongs to $\mathsf{C}_{\mathsf{AGE}}(\bm{\alpha},k)$ iff $\bm{c}_{\setminus k}\geq\bm{0}$ and

[TABLE]

The system of constraints given by (2) is crucially jointly convex in $\bm{c}$ and the auxiliary variable $\bm{\nu}$. The set defined by the sum of all AGE cones

[TABLE]

is therefore efficiently representable.

The SAGE cone as defined above applies to signomials, but a similar construction exists for certifying global nonnegativity of polynomials [SAGE2]. Formally, we say that $f=\operatorname{Poly}(\bm{\alpha},\bm{c})$ is an AGE polynomial if it is nonnegative over $\mathbb{R}^{n}$, and if $f(\bm{x})$ contains at most one term $c_{i}\bm{x}^{\bm{\alpha}_{i}}$ that is not a monomial square. In conic form this writes as

[TABLE]

and such AGE cones naturally give rise to

[TABLE]

The SAGE polynomial cone can also be described by an appropriate reduction to the SAGE signomial cone. For a nonnegative $m\times n$ integer matrix $\bm{\alpha}$ and a vector $\bm{c}$ in $\mathbb{R}^{m}$, we define the set of signomial representative coefficient vectors as

[TABLE]

The name “signomial representative” derives from the fact that if $\bm{\hat{c}}$ belongs to $\mathrm{SR}(\bm{\alpha},\bm{c})$, then nonnegativity of the signomial $\operatorname{Sig}(\bm{\alpha},\bm{\hat{c}})$ would evidently imply nonnegativity of the polynomial $\operatorname{Poly}(\bm{\alpha},\bm{c})$ (see Section 5.1 of [SAGE2]). The set $\mathrm{SR}(\bm{\alpha},\bm{c})$ is useful because a constraint of the form “$\bm{\hat{c}}$ belongs to $\mathrm{SR}(\bm{\alpha},\bm{c})$” is jointly convex in $\bm{\hat{c}}$ and $\bm{c}$. Lemma 19 of [SAGE2] proves that the SAGE polynomial cone defined by Equation 5 is equivalently given by

[TABLE]

The generalization of SAGE polynomials considered in this article benefits from both the Sum-of-AGE-function and signomial-representative viewpoints of ordinary SAGE polynomials.

Lastly we consider Sums-of-Squares (SOS) polynomials. A polynomial $f$ is said to be SOS if it can be written in the form $f=\sum_{i=1}^{m}f_{i}^{2}$ for appropriate polynomials $f_{i}$. In the context of polynomial optimization, one usually parameterizes the SOS cone by a number of variables $n$ and a maximum degree $2d$; this cone can be represented as

[TABLE]

where $L^{n}_{d}:\mathbb{R}^{n}\to\mathbb{R}^{{n+d\choose d}}$ is the map from a vector $\bm{x}$ to the vector of all monomials of degree at-most-$d$ evaluated at $\bm{x}$. The connection between SOS-representability and semidefinite programming was first observed by Shor [shor], and was subsequently developed by Parrilo [ParriloPhD] and Lasserre [Lasserre2001].

2.3 Strengthening dual bounds in nonnegativity relaxations

A common method for strengthening dual problems is to introduce redundant constraints to the primal problem, particularly by taking products of existing constraint functions. As an example of this principle in action, consider the toy polynomial optimization problem

[TABLE]

One may verify that $(f,g)^{\mathrm{L}}_{\mathbb{R}}=-\infty$, but by adding the single redundant constraint $(1-x)(1+x)\geq 0$, we can certify a dual bound $-1\leq(f,g)_{\mathbb{R}}^{\star}$.

A more subtle method is to reconsider what is meant by “dual variables.” For the Lagrange dual problem we use scalars $\lambda_{i}\geq 0$, however it would be just as valid to have $\lambda_{i}$ be a function, provided that it was nonnegative over $\mathbb{R}^{n}$. Such a method is well-suited to our nonnegativity-based relaxations of the dual problem. The following toy signomial program illustrates the utility of this approach

[TABLE]

Again the Lagrange dual problem returns a bound of $-\infty$, but by considering $\lambda_{i}$ of the form $\lambda_{i}(x)=\eta_{i}\exp(x)$ with $\eta_{i}\geq 0$, the resulting dual bound is $-4\leq(f,g)_{\mathbb{R}}^{\star}$.

Our third method for strengthening dual bounds only becomes relevant when working with strict inner-approximations of nonnegativity cones. For two functions $w,f$ with $w$ positive definite, it is clear that $f$ is nonnegative if and only if the product $w\cdot f$ is nonnegative. The method of modulation is to choose a generic positive-definite function $w$ so that if $f$ fails a particular test for nonnegativity (say, being SOS, or being SAGE), there is still a chance that the product $w\cdot f$ passes a test for nonnegativity. Indeed, modulation is a crucial tool for computing successive bounds for unconstrained problems

[TABLE]

Suppose for example that $f$ is a signomial over exponents $\bm{\alpha}$; then for $w=\operatorname{Sig}(\bm{\alpha},\bm{1})$ we can compute a non-decreasing sequence of lower bounds

[TABLE]

Under appropriate conditions on $\bm{\alpha}$ (c.f. [SAGE1]), these lower bounds converge to $f^{\star}_{\mathbb{R}^{n}}$ as $\ell$ goes to infinity. From an implementation perspective, the constraint that “$\psi(\gamma)\doteq w^{\ell}(f-\gamma)$ is SAGE” is tractable because the coefficient vector of $\psi(\gamma)$ is an affine function of $\gamma$.

Modulation can similarly be applied to constrained optimization. Suppose that $\mathcal{L}(\bm{x},\bm{\lambda})$ is the Lagrangian for Problem 1.1, and refer to the function $\bm{x}\mapsto\mathcal{L}(\bm{x},\bm{\lambda})$ as $\mathcal{L}(\bm{\lambda})$. Then rather than requiring that “$\mathcal{L}(\bm{\lambda})-\gamma$ is SAGE”, one could require that “$\psi(\gamma,\bm{\lambda})\doteq w^{\ell}(\mathcal{L}(\bm{\lambda})-\gamma)\text{ is SAGE}$.” This would increase the size of the feasible set for variables $\gamma$ and $\bm{\lambda}$, and remain tractable due to the affine dependence of $\psi(\gamma,\bm{\lambda})$ on $\gamma$ and $\bm{\lambda}$. Such modulation leads to a non-decreasing sequence of bounds which converge to $(f,g)_{\mathbb{R}^{n}}^{\mathrm{L}}$ under suitable conditions.

2.4 Partial dualization

A partial dual problem is what results when the set “$X$” in Problem 1.1 is a proper subset of $\mathbb{R}^{n}$. In this case the natural generalization of the Lagrange dual is

[TABLE]

The technique of partial dualization refers to the deliberate choice to restrict the Lagrangian to $X=\{\bm{x}:g_{i}(\bm{x})\geq 0\text{ for all }i\text{ in }\mathcal{I}\}$ for some $\mathcal{I}\subset[k]$, even when the constraints $\{g_{i}\}_{i\in\mathcal{I}}$ are of a functional form that is permitted in the Lagrangian. Note that in the extreme case with $X=\{\bm{x}:g(\bm{x})\geq\bm{0}\}$, we are certain to have $(f,0)^{\mathrm{d}}_{X}=(f,g)^{\star}_{\mathbb{R}^{n}}$ – in this way, partial dualization provides a mechanism to completely eliminate duality gaps.

Before getting into how SAGE certificates integrate with partial dualization, it is worth considering a simple example which combines partial dualization and nonnegativity certificates. Suppose we want to minimize a univariate polynomial $f$ over an interval $[a,b]$, subject to a single polynomial equality constraint $g(x)=0$. In this case we could form a Lagrangian $\mathcal{L}(x,\mu)=f(x)-\mu g(x)$ with $\mu\in\mathbb{R}$, and find the largest constant $\gamma$ so that $\mathcal{L}(x,\mu)-\gamma$ was nonnegative over $x\in[a,b]$. A well-known result in real algebraic geometry is that a degree-$d$ polynomial “$p$” is nonnegative over an interval $[a,b]$ if and only if $p$ can be written as $p(x)=s(x)^{2}+h_{[a,b]}(x)t(x)^{2}$, where $h_{[a,b]}(x)=(b-x)(x-a)$, and $s$, $t$ are polynomials of degree at most $d$ and $d-1$ respectively [Powers2000]. Therefore the partial dual problem

[TABLE]

can be framed as an SOS relaxation

[TABLE]

Our last key concept is how partial dualization manifests in the dual of the dual. To develop this idea, consider $f=\operatorname{Poly}(\bm{\alpha},\bm{c})$ with $\bm{\alpha}_{1}=\bm{0}$, along with a set $X\subset\mathbb{R}^{n}$. The problem of computing $f_{X}^{\star}\doteq\inf_{\bm{x}\in X}f(\bm{x})$ has the following convex formulation

[TABLE]

Of course, the above problem is intractable unless $\bm{\alpha}$ and $X$ satisfy very special conditions. In spite of the possible intractability, we can still compute the dual problem by applying standard rules of conic duality. The result of this process is

[TABLE]

where $\mathsf{C}_{\mathsf{NNP}}(\bm{\alpha},X)^{\dagger}$ is the dual cone to $\mathsf{C}_{\mathsf{NNP}}(\bm{\alpha},X)$. This second problem is what we mean by “the dual of the dual.” It appears prominently in the literature on polynomial optimization, where it is usually referred to as a moment relaxation [Lasserre2001]. The term “moment relaxation” derives from the fact that $\mathsf{C}_{\mathsf{NNP}}(\bm{\alpha},\mathbb{R}^{n})^{\dagger}$ is the smallest closed convex cone containing the vectors

[TABLE]

and by thinking of a convex hull as computing expectations $\mathbb{E}_{\bm{x}\sim F}[(\bm{x}^{\bm{\alpha}_{1}},\ldots,\bm{x}^{\bm{\alpha}_{m}})]$, where $F$ is a probability measure over $\mathbb{R}^{n}$. One can similarly understand the dual of a nonnegativty-based partial-dual problem in terms of probability and moment relaxations. When $X$ as a proper subset of $\mathbb{R}^{n}$, the convex hull of $X^{\bm{\alpha}}$ can be framed as the set of all vector-valued expectations $\mathbb{E}_{\bm{x}\sim F}[(\bm{x}^{\bm{\alpha}_{1}},\ldots,\bm{x}^{\bm{\alpha}_{m}})]$, where $F$ is a probability measure over $X$. In this way, the “dual” of partial dualization can be understood in terms of conditional moments.

3 Conditional SAGE certificates for signomials

In this section we show how SAGE certificates for signomial nonnegativity can fully leverage partial dualization, in the sense that any efficiently representable convex set $X$ gives rise to a parameterized and efficiently representable “$X$-SAGE” nonnegativity cone. The efficient representation of the $X$-SAGE cones (which we often call “conditional SAGE cones”) leads to a practical, principled approach for solving and approximating a range of nonconvex signomial optimization problems. In this regard the most common sets $X$ are of the form $\{\bm{x}:g(\bm{x})\leq\bm{1}\}$ for signomials $g_{i}$ with all nonnegative coefficients. An algorithm for solution recovery, and two worked examples are provided.

3.1 The conditional SAGE signomial cones

Definition 1** (Conditional AGE signomial cones).**

For a matrix $\bm{\alpha}$ in $\mathbb{R}^{m\times n}$, a subset $X$ of $\mathbb{R}^{n}$, and an index $k$ in $[m]$, the $k^{\text{th}}$ AGE cone with respect to $\bm{\alpha},X$ is

[TABLE]

Definition 2** ($X$-SAGE signomials).**

If the vector $\bm{c}$ belongs to

[TABLE]

then $f=\operatorname{Sig}(\bm{\alpha},\bm{c})$ is an $X$-SAGE signomial.

Conditional SAGE cones are order-reversing with respect to the second argument. That is, if $X_{2}\subset X_{1}\subset\mathbb{R}^{n}$, then $\mathsf{C}_{\mathsf{SAGE}}(\bm{\alpha},X_{1})\subset\mathsf{C}_{\mathsf{SAGE}}(\bm{\alpha},X_{2})$ for all $\bm{\alpha}$ in $\mathbb{R}^{m\times n}$. Note that $\mathsf{C}_{\mathsf{SAGE}}(\bm{\alpha},X)$ is defined for arbitrary $X\subset\mathbb{R}^{n}$, including nonconvex sets, and convex sets which admit no efficient description. Moreover, as a mathematical object, $\mathsf{C}_{\mathsf{SAGE}}(\bm{\alpha},X)$ does not depend on the representation of $X$.

In practice we need conditions on $X$ in order to optimize over $\mathsf{C}_{\mathsf{SAGE}}(\bm{\alpha},X)$. But before we get to those, it is worth mentioning some abstract results concerning optimization. For $f=\operatorname{Sig}(\bm{\alpha},\bm{c})$ with $\bm{\alpha}_{1}=\bm{0}$, define

[TABLE]

so that $f^{\mathrm{SAGE}}_{X}\leq f_{X}^{\star}\doteq\inf\{f(\bm{x}):\bm{x}\text{ in }X\}$.

Theorem 3**.**

If $\bm{c}\geq\bm{0}$, then $f=\operatorname{Sig}(\bm{\alpha},\bm{c})$ has $f^{\mathrm{SAGE}}_{X}=f_{X}^{\star}$ for all $X\subset\mathbb{R}^{n}$.

Proof.

Let $\bm{e}_{1}=(1,0,\ldots,0)$. The signomial $\tilde{f}=\operatorname{Sig}(\bm{\alpha},\bm{c}-f_{X}^{\star}\bm{e}_{1})$ is nonnegative over $X$, and its coefficient vector $\bm{c}-f_{X}^{\star}\bm{e}_{1}$ contains at most one negative entry. This implies that $\tilde{f}$ is $X$-AGE, and hence $X$-SAGE. ∎

Theorem 4**.**

If $X$ is bounded, then $f^{\mathrm{SAGE}}_{X}>-\infty$ for every signomial $f$.

Proof.

If $X$ is empty then the result follows by verifying that $\mathsf{C}_{\mathsf{SAGE}}(\bm{\alpha},X)=\mathbb{R}^{m}$. Consider the case when $X$ is nonempty. In this situation it suffices to prove the result for all $f$ of the form $f(\bm{x})=c\exp(\bm{a}\cdot\bm{x})$ where $c\neq 0$ and $\bm{a}$ belongs to $\mathbb{R}^{1\times n}$. Fixing such $c$, $\bm{a}$, the boundedness of $X$ implies the existence of $L\neq 0$ with $\tilde{f}(\bm{x})=c\exp(\bm{a}^{\intercal}\bm{x})+L$ nonnegative over $\bm{x}$ in $X$ and $cL<0$. Since $\tilde{f}$ is nonnegative over $X$ and contains exactly one negative coefficient, we have that $f^{\mathrm{SAGE}}_{X}\geq-L$. ∎

Corollary 5** (See [SAGE2]).**

Let $X\subset\mathbb{R}^{n}$ be arbitrary. If $\bm{c}$ is a vector in $\mathsf{C}_{\mathsf{SAGE}}(\bm{\alpha},X)$ with nonempty $\mathcal{N}=\{i:c_{i}<0\}$, then there exist vectors $\{\bm{c}^{(i)}\}_{i\in\mathcal{N}}$ satisfying

[TABLE]

Proof.

This is simply the statement of Theorem 2 from [SAGE2], which was proven for ordinary SAGE cones, i.e. with $X=\mathbb{R}^{n}$. The entire proof of that theorem (including Lemmas 6 and 7 of [SAGE2]) extends to conditional SAGE cones simply by replacing references to “$\mathsf{C}_{\mathsf{AGE}}(\bm{\alpha},i)$” and “$\mathsf{C}_{\mathsf{SAGE}}(\bm{\alpha})$” with “$\mathsf{C}_{\mathsf{AGE}}(\bm{\alpha},i,X)$” and “$\mathsf{C}_{\mathsf{SAGE}}(\bm{\alpha},X)$” respectively. ∎

In order to reliably optimize over $\mathsf{C}_{\mathsf{SAGE}}(\bm{\alpha},X)$, we need $X$ to be a tractable convex set. This is essentially the only requirement on $X$, as is shown by the following theorem.

Theorem 6**.**

For a matrix $\bm{\alpha}$ in $\mathbb{R}^{m\times n}$, an index $i$ in $[m]$, and a convex set $X\subset\mathbb{R}^{n}$ with support function $\sigma_{X}(\bm{\lambda})\doteq\sup_{\bm{x}\in X}\bm{\lambda}^{\intercal}\bm{x}$, we have

[TABLE]

Proof.

Let $\delta_{X}$ denote the indicator function of $X$, taking values

[TABLE]

A vector $\bm{c}$ with $\bm{c}_{\setminus i}\geq\bm{0}$ belongs to $\mathsf{C}_{\mathsf{AGE}}(\bm{\alpha},i,X)$ if and only if

[TABLE]

for $\ell=m-1$, $\bm{W}=[\bm{\alpha}_{\setminus i}-\bm{1}\bm{\alpha}_{i}]\in\mathbb{R}^{\ell\times n}$, $\bm{\tilde{c}}=\bm{c}_{\setminus i}\in\mathbb{R}^{\ell}$, and $L=c_{i}$.

The dual to the above optimization problem is easily calculated by applying Fenchel duality (c.f. [Borwein2006]); the result of this process is

[TABLE]

When $X$ is nonempty, one may verify that the hypothesis of Corollary 3.3.11 of [Borwein2006] (concerning strong duality) hold for the primal-dual pair (7)-(8). In particular, $p^{\star}\geq-L$ holds if and only if $-d^{\star}\leq L$, and the dual problem attains an optimal solution whenever finite. When $X$ is empty, it is clear that $p^{\star}=+\infty$, and by taking both $\bm{\lambda}$ and $\bm{\nu}$ as zero vectors, we have $d^{\star}=+\infty$. The result follows. ∎

Theorem 6 is stated in terms of support functions for maximum generality. From an implementation perspective, it is useful to assume a representation of $X$. For example, if $X=\{\bm{x}\,:\,\bm{A}\bm{x}+\bm{b}\in K\}$ for a matrix $\bm{A}$, a vector $\bm{b}$, and a convex cone $K$, then weak duality ensures

[TABLE]

An upper bound on the support function is all we need to construct an inner-approximation of a given AGE cone. For all $X=\{\bm{x}\,:\,\bm{A}\bm{x}+\bm{b}\in K\}$, we have

[TABLE]

If there exists an $\bm{x}_{0}$ so that $\bm{A}\bm{x}_{0}+\bm{b}$ belongs to the relative interior of $K$, then by Slater’s condition the reverse inclusion in the preceding expression also holds.

3.2 Dual perspectives and solution recovery

Here we discuss how dual SAGE relaxations can be used to recover optimal and near-optimal solutions to signomial programs of the form (1). For concreteness, we state the simplest such relaxation here. Let $f$, $\{g_{i}\}_{i=1}^{k_{1}}$ and $\{\phi_{i}\}_{i=1}^{k_{2}}$ be signomials over exponents $\bm{\alpha}$, with $\bm{\alpha}_{1}=\bm{0}$. If $\bm{c}$ is the coefficient vector of $f$, and the rows of $\bm{G}\in\mathbb{R}^{k_{1}\times m}$, $\bm{\Phi}\in\mathbb{R}^{k_{2}\times m}$ specify coefficient vectors of $g_{i}$, $\phi_{i}$ respectively, then

[TABLE]

is a convex relaxation of Problem 1. It is readily verified that if $\bm{v}^{\star}$ is an optimal solution to (9) and $\bm{v}^{\star}=\exp(\bm{\alpha}\bm{x})$ for some $\bm{x}$ in $X$, then $\bm{x}$ is optimal for (1).

The prospect of inverting the moment-type vector $\bm{v}^{\star}$ to obtain a feasible point $\bm{x}\in X$ drives our interest in understanding the dual cone $\mathsf{C}_{\mathsf{SAGE}}(\bm{\alpha},X)^{\dagger}$. By standard rules in convex analysis, the dual SAGE cone is given by

[TABLE]

An expression for the dual AGE cones can be recovered from Theorem 6. Using $\operatorname{co}X=\{(\bm{x},t)\,:\,t>0,~{}\bm{x}/t\in X\}$ to denote the cone over $X$, the usual conic-duality calculations yield

[TABLE]

The auxiliary variables “$\bm{z}$” appearing in the representation for the dual $X$-AGE cones are a powerful tool for solution recovery. If $\bm{z},\bm{v}$ satisfy the constraints in (10) with $v_{i}>0$, then $\bm{x}\doteq\bm{z}/v_{i}$ belongs to $X$. Additionally, if the inequality constraints involving the logarithm are binding and we meet the normalization condition $v_{i}=\exp(\bm{\alpha}_{i}\cdot\bm{x})$, then we will have $\exp(\bm{\alpha}\bm{x})=\bm{v}$.

These observations form the core – but not the entirety – of our solution recovery algorithm. Depending on solver behavior and dimension-reduction techniques used to simplify a SAGE relaxation, it is possible that $\bm{v}=\exp(\bm{\alpha}\bm{x})$ for some $\bm{x}$ in $X$, and yet no dual AGE cone generated this value. Therefore it is also prudent to solve a constrained least-squares problem to find $\bm{x}$ in $X$ with $\bm{\alpha}\bm{x}$ near $\log\bm{v}$.

Assuming that Equation 10 is used to represent the dual AGE cones, the runtime of Algorithm 1 is dominated by Line 10. The runtime of Line 10, in turn, should be substantially smaller than the time required to compute $\bm{v}\in\mathsf{C}_{\mathsf{SAGE}}(\bm{\alpha})^{\dagger}$ which is optimal for an appropriate SAGE relaxation.

Note how Algorithm 1 implicitly assumes that $\bm{v}$ is elementwise positive. For numerical reasons, this will be the case in practice. At a theoretical level, if $\bm{v}$ contains some component $v_{i}=0$, then there is no $\bm{x}$ in $\mathbb{R}^{n}$ which could attain $v_{i}=\exp(\bm{\alpha}_{i}\cdot\bm{x})$, and so solution recovery is not a well-posed problem in such situations. Note also how in the important (and possibly nontrivial) case with $k_{1}=k_{2}=0$, Algorithm 1 always returns at least one feasible solution.

In the authors experience it is useful to take solutions generated from Algorithm 1 and pass them to a local solver as initial conditions. This additional step is especially worthwhile when there is a gap between a SAGE bound and a problem’s true optimal value, or when solution recovery to the desired precision of $\epsilon_{\text{ineq}},\epsilon_{\text{eq}}$ fails. The term “Algorithm 1L” henceforth refers to the use of Algorithm 1, followed by local-solver solution refinement. For the examples in this article, the authors use Powell’s COBYLA solver for the solution refinement [Powell1994].222A FORTRAN implementation is accessible through SciPy’s optimize submodule. The arguments we pass to that FORTRAN implementation are RHOBEG=1, RHOEND$=10^{-7}$, and MAXFUN$=10^{5}$.

3.3 A first worked example

The following problem has appeared in many articles concerning algorithms for signomial programming [Shen2004, Shen2006, Qu2007, Shen2008, HSC2014].

[TABLE]

This problem (“Example 1”) is an excellent candidate for conditional SAGE relaxations, because each of the seven constraints defines an efficiently representable convex set. Constraint functions $g_{2:7}$ can be can be represented with six linear inequalities, and the constraint function $g_{1}$ can be represented with three exponential cones and one linear inequality. As a separate matter, Example 1 is interesting because the Lagrange dual problem performs poorly: regardless of how many products we take of existing constraint functions $g_{i}$, the $-5\exp(-x_{2})$ term in the objective will cause Lagrangians $f-\sum_{I}\lambda_{I}\prod_{j\in I}g_{j}$ to be unbounded below for all values of dual variables $\lambda_{I}\geq 0$.

Now we describe how SAGE relaxations fare for Example 1. We begin by setting $X=\{\bm{x}:g_{1:7}(\bm{x})\}$; since $X$ is bounded, Theorem 4 tells us that $f_{X}^{\mathrm{SAGE}}$ is finite. The dual SAGE relaxation can be solved with MOSEK on Machine $\bm{\mathsf{L}}$ in 0.01 seconds, and provides us with a lower bound $f_{X}^{\mathrm{SAGE}}=-147.86\leq f_{X}^{\star}$. By running Algorithm 1 on the dual solution, we recover

[TABLE]

From this solution, we know that the bound obtained from the SAGE relaxation is within 0.13% relative error of the true optimal value. The ability to recover near-optimal solutions even in the presence of a gap $f_{X}^{\mathrm{SAGE}}<f_{X}^{\star}$ can be attributed to how our solution recovery algorithm differs from traditional “moment” techniques. As it happens, the point $\bm{x}^{\star}$ recovered from Algorithm 1 is actually an optimal solution to Example 1. In order to certify this fact, we need stronger SAGE relaxations. Table 1 shows the results of these relaxations, using the minimax-free hierarchy described in Section 3.4.

3.4 Reference hierarchies for signomial programming

This section gives a particular set of choices regarding SAGE-based hierarchies for signomial programming. Because SAGE certificates are sparsity-preserving, one must be careful when describing relaxations which use nonconstant Lagrange multipliers, or positive-definite modulators. In particular, when we say “$f$ and $g_{i}$ are signomials over exponents $\bm{\alpha}$,” we mean that $\{\bm{x}\mapsto\exp(\bm{\alpha}_{j}\cdot\bm{x})\}_{j=1}^{m}$ is the smallest monomial basis spanning all linear combinations of $f$, $g_{i}$, and the function that is identically equal to one.

First we describe a SAGE-based hierarchy that does not make use of the minimax inequality. This could be understood as a hierarchy for unconstrained optimization, but really applies whenever minimizing a signomial over a tractable convex set $X\subset\mathbb{R}^{n}$. In the event that we cannot certify nonnegativity $f-\gamma$ with $\gamma=f_{X}^{\star}$, we can use modulators as described in Section 2.3 to improve the largest SAGE-certifiable bound on $f$. Formally, for a signomial $f$ over exponents $\bm{\alpha}$, a nonnegative integer $\ell$, and a tractable convex set $X$, the level-$\ell$ SAGE relaxation for $f_{X}^{\star}$ is

[TABLE]

The special case with $\ell=0$ was introduced earlier in this section as “$f_{X}^{\mathrm{SAGE}}$.”

Now we consider the case with functional constraints; let $f$, $g_{i}$, and $\phi_{i}$ be signomials over exponents $\bm{\alpha}$. SAGE relaxations for the problem of computing $(f,g,\phi)_{X}^{\star}$ are indexed by three integer parameters: $p$, $q$, and $\ell$. Starting from $p\geq 0$ and $q\geq 1$, define $\bm{\alpha}[p]$ as the matrix of exponent vectors for $\operatorname{Sig}(\bm{\alpha},\bm{1})^{p}$, and define $g[q]$ as the set of all products of at-most-$q$ elements of $g$ (similarly define $\phi[q]$). The SAGE relaxation for $(f,g,\phi)_{X}^{\star}$ at level $(p,q,\ell)$ is then

[TABLE]

The decision variables in (11) are $\gamma\in\mathbb{R}$, the coefficient vectors of $\{s_{h}\}_{h\in g[q]}$, and the coefficient vectors of $\{z_{h}\}_{h\in\phi[q]}$. The most basic level of this hierarchy is $(p,q,\ell)=(0,1,0)$. This corresponds to using scalar Lagrange multipliers ($s_{h}\geq 0$ and $z_{h}\in\mathbb{R}$), the original constraints ($g[0]=g,~{}\phi[0]=\phi$), and modulating the Lagrangian by the signomial that is identically equal to 1. Note that when $p>0$, the Lagrange multipliers $s_{h}$ are required to be nonnegative only over $X$, rather than over the whole of $\mathbb{R}^{n}$.

Once an appropriate SAGE relaxation has been solved, there is the matter of attempting to recover a solution from the dual problem. Oftentimes a SAGE relaxation produces a tight bound on $(f,g,\phi)_{X}^{\star}$, and yet no feasible solution can be recovered from Algorithm 1 with reasonable values of $\epsilon_{\text{eq}}$. Thus we also suggest that one eliminate equality constraints through substitution of variables, when possible. When it is not possible to eliminate all equality constraints, we recommend allowing large violations of equality constraints in Algorithm 1 (e.g. $\epsilon_{\mathrm{eq}}=1.0$), and passing the returned values as near-feasible points to a local solver in the manner of Algorithm 1L.333COBYLA is an excellent example of a solver which suppports infeasible starts. This principle also extends to allowing large values of $\epsilon_{\text{ineq}}$ prior to solution-refinement, however the authors find that this is usually not necessary.

3.5 A second worked example

This section’s example can be found in the 1976 PhD thesis of James Yan [Yan1976], where it illustrates signomial programming in the service of structural engineering design. This problem is notable because it is nonconvex even when written in exponential form. Such signomial programs have received limited attention in the engineering design optimization community, largely due to a lack of reliable methods for solving them. We restate the problem here (as Example 2) in geometric form.444The objective and inequality constraint functions are multiplied by $10^{4}$ for numerical reasons; see equation environment (6.15) on page 106 of [Yan1976] for the original problem statement.

[TABLE]

Let $X\subset\mathbb{R}^{4}$ be the feasible set cut out by the eight bound constraints in Example 2. With an $X$-SAGE relaxation where all constraints appear in the Lagrangian, we obtain $(f,g,\phi)_{X}^{(0,1,0)}=14.1423$ in 0.04 seconds of solver time. This bound is very close to the optimal value claimed by Yan [Yan1976]. However, Algorithm 1 only returns candidate solutions “$\bm{x}$” with equality constraint violations $\phi(\bm{x})\approx 70$.

In an effort to improve our chances of solution recovery, we use the equality constraint to define the value $P\leftarrow 70.7107A_{3}/(A_{3}+A_{1})$. After clearing the denominator $(A_{3}+A_{1})$ for inequality constraints involving $P$, we obtain a signomial program (in geometric-form) in only the variables $A_{1},A_{2},A_{3}$. We solve a level-$(0,1,0)$ dual conditional SAGE relaxation for this signomial program, and exponentiate the result of Algorithm 1 to recover

[TABLE]

This solution is feasible up to machine precision, and attains objective matching the 14.142300 SAGE bound. The entire process of solving the SAGE relaxation and recovering the optimal solution takes less than 0.05 seconds on Machine $\bm{\mathsf{W}}$.

3.6 Remarks on “geometric-form” signomial programming

By now we have seen signomial programs in both exponential and geometric forms. The authors have hitherto preferred the exponential form, primarily because it allows us to build upon the substantial theories of convex analysis and convex optimization. However it is important to acknowledge that from an applications perspective, it is far more common to express signomial programs in geometric form. Here we briefly present a geometric-form parameterization of conditional SAGE certificates for signomial nonnegativity – both in effort to appeal to those who are accustomed to working with geometric-form signomial programs, and as a prelude to our discussion on conditional SAGE polynomials.

Geometric form signomials $f(\bm{x})=\sum_{i=1}^{m}c_{i}\bm{x}^{\bm{\alpha}_{i}}$ are defined at points $\bm{x}$ in $\mathbb{R}^{n}_{++}$, and so it only makes sense to discuss conditional nonnegativity cones for signomials over sets $X\subset\mathbb{R}^{n}_{++}$. Henceforth, define

[TABLE]

for matrices $\bm{\alpha}$ in $\mathbb{R}^{m\times n}$ and sets $X$ contained in $\mathbb{R}^{n}_{++}$. By applying the change of variables $\bm{x}\mapsto\exp\bm{y}$ and considering the subsequent change of domain $X\mapsto\log X$, one may verify that $\mathsf{C}_{\mathsf{NNS}}^{\mathsf{GEOM}}(\bm{\alpha},X)=\mathsf{C}_{\mathsf{NNS}}(\bm{\alpha},\log X)$. Thus for $X\subset\mathbb{R}^{n}_{++}$, one arrives naturally at the definition

[TABLE]

From here it should be easy to deduce various corollaries for $\mathsf{C}_{\mathsf{SAGE}}^{\mathsf{GEOM}}(\bm{\alpha},X)$, by appealing to results from Section 3.1. The most important such result is that one can efficiently optimize over $\mathsf{C}_{\mathsf{SAGE}}^{\mathsf{GEOM}}(\bm{\alpha},X)$ whenever $\log X$ is a convex set for which the epigraph of the support function is efficiently representable.

4 Conditional SAGE certificates for polynomials

In the previous section we saw how conditional SAGE certificates for signomial nonnegativity are inextricably linked to convex duality. Here we show how the broader idea of conditional SAGE certificates can extend to polynomials. In this context it is not convexity of $X$ that determines when an $X$-SAGE polynomial cone is tractable, but rather convexity of an appropriate logarithmic transform of $X$.

The organization of this section is similar to that of Section 3. Definitions, representations, and other basic theorems for the conditional SAGE polynomial cones are given in Section 4.1. Section 4.2 covers solution recovery from dual SAGE relaxations, and Section 4.3 provides a worked example with special focus on solution recovery. Section 4.4 describes reference hierarchies for optimization with conditional SAGE polynomial cones: one based on the minimax inequality, and one that is “minmax free.” Section 4.5 applies various manifestations of the minimax hierarchy to an example problem.

4.1 The conditional SAGE polynomial cones

We call $f=\operatorname{Poly}(\bm{\alpha},\bm{c})$ an $X$-AGE polynomial if it is nonnegative over $X$, and $f(\bm{x})$ contains at most one term $c_{i}\bm{x}^{\bm{\alpha}_{i}}$ which is negative for some $\bm{x}$ in $X$.

Definition 7** (Conditional AGE polynomial cones).**

For $\bm{\alpha}$ in $\mathbb{N}^{m\times n}$, a subset $X$ of $\mathbb{R}^{n}$, and an index $i$ in $[m]$, the $i^{\text{th}}$ AGE polynomial cone with respect to $\bm{\alpha},X$ is

[TABLE]

Let us work through some consequences of the definition. For starters, if $\bm{x}^{\bm{\alpha}_{j}}$ takes on positive and negative values as $\bm{x}$ varies over $X$, then $c_{j}=0$ whenever $\bm{c}\in\mathsf{C}_{\mathsf{AGE}}^{\mathsf{POLY}}(\bm{\alpha},i,X)$ and $i\neq j$. Note that $\bm{x}^{\bm{\alpha}_{j}}$ can only take on both positive and negative values when $\bm{\alpha}_{j}$ does not belong to the even integer lattice. If $X$ contains an open ball around the origin, then $\bm{x}^{\bm{\alpha}_{j}}$ takes on both positive and negative values if and only if $\bm{\alpha}_{j}$ does not belong to the even integer lattice. Thus Definition 7 agrees with the definition of ordinary AGE polynomial cones, as proposed in [SAGE2] and as restated in Equation 4. Another important case is when $X$ is a subset of the nonnegative orthant. This point is addressed in some detail later in this section; as a preliminary remark, we note that by considering the connection between polynomials and geometric-form signomials, one can easily see that if $X\subset\mathbb{R}^{n}_{++}$ then $\mathsf{C}_{\mathsf{AGE}}^{\mathsf{POLY}}(\bm{\alpha},i,X)=\mathsf{C}_{\mathsf{AGE}}(\bm{\alpha},i,\log X)$. With these facts in mind, we define the conditional SAGE polynomial cone in the natural way.

Definition 8** ($X$-SAGE polynomials).**

If the vector $\bm{c}$ belongs to

[TABLE]

then we say that $f=\operatorname{Poly}(\bm{\alpha},\bm{c})$ is an $X$-SAGE polynomial.

Many of our earlier theorems for signomials apply to $X$-SAGE polynomials without any special assumptions on $X$. For example, it is easy to show that Theorem 4 applies to conditional SAGE polynomials: if $X$ is bounded, then $f=\operatorname{Poly}(\bm{\alpha},\bm{c})$ with $\bm{\alpha}_{1}=\bm{0}$ has

[TABLE]

Corollary 5 likewise extends to polynomials. Other than substituting AGE signomial cones with AGE polynomial cones, the only difference is that $\mathcal{N}$ becomes $\mathcal{N}=\{i:c_{i}\bm{x}^{\bm{\alpha}_{i}}<0\text{ for some }\bm{x}\text{ in }X\}$.

Now we turn to representation of SAGE polynomial cones. By applying a simple continuity argument one can show that if $X=\operatorname{cl}X^{\circ}\subset\mathbb{R}^{n}_{+}$ – where $X^{\circ}$ is the interior of $X$ – then $\mathsf{C}_{\mathsf{SAGE}}^{\mathsf{POLY}}(\bm{\alpha},X)=\mathsf{C}_{\mathsf{SAGE}}(\bm{\alpha},\log X^{\circ})$. This claim is strengthened slightly and made more explicit through the following theorem.

Theorem 9**.**

Suppose $X\subset\mathbb{R}^{n}_{+}$ is representable as $X=\operatorname{cl}\{\bm{x}:\bm{0}<\bm{x},H(\bm{x})\leq\bm{1}\}$ for a continuous map $H:\mathbb{R}^{n}\to\mathbb{R}^{r}$. Then for $Y=\{\bm{y}:H(\exp\bm{y})\leq\bm{1}\}$, we have $\mathsf{C}_{\mathsf{SAGE}}^{\mathsf{POLY}}(\bm{\alpha},X)=\mathsf{C}_{\mathsf{SAGE}}(\bm{\alpha},Y)$.

The proof of Theorem 9 is straightforward, and hence omitted. A more sophisticated result concerns when $X$ is not contained in any particular orthant, but nevertheless possesses a certain sign-symmetry.

Theorem 10**.**

Suppose $X\subset\mathbb{R}^{n}$ is representable as $X=\operatorname{cl}\{\bm{x}:\bm{0}<|\bm{x}|,H(|\bm{x}|)\leq\bm{1}\}$ for a continuous map $H:\mathbb{R}^{n}\to\mathbb{R}^{r}$. Then for $Y=\{\bm{y}:H(\exp\bm{y})\leq\bm{1}\}$, we have

[TABLE]

By combining Theorem 6 with Theorems 9 and 10, we know that there exist a range of sets $X$ for which optimization over $X$-SAGE polynomials is tractable. There remains the potentially nontrivial task of formulating a problem so that one of these theorems provides an efficient representation of $\mathsf{C}_{\mathsf{SAGE}}^{\mathsf{POLY}}(\bm{\alpha},X)$; important examples of when this is possible include constraints such as

[TABLE]

where $a>0$ is a fixed constant.

Proof of Theorem 10.

Suppose that $\bm{c}$ in $\mathsf{C}_{\mathsf{SAGE}}^{\mathsf{POLY}}(\bm{\alpha},X)$ admits the decomposition $\bm{c}=\sum_{i=1}^{m}\bm{c}^{(i)}$, where $\bm{c}^{(i)}$ belongs to the $i^{\text{th}}$ AGE polynomial cone with respect to $\bm{\alpha},X$. Define $\{\bm{\tilde{c}}^{(i)}\}_{i=1}^{m}$ as follows

[TABLE]

By the invariance of $X$ under reflection about hyperplanes $\{\bm{x}:x_{j}=0\}$, and continuity of polynomials, we have that

[TABLE]

The signomials $\operatorname{Sig}(\bm{\alpha},\bm{\tilde{c}}^{(i)})$ are thus nonnegative over $Y=\{\bm{y}:H(\exp\bm{y})\leq 1\}$, and posses at most one negative coefficient. This implies that $\bm{\tilde{c}}\doteq\sum_{i=1}^{m}\bm{\tilde{c}}^{(i)}$ belongs to $\mathsf{C}_{\mathsf{SAGE}}(\bm{\alpha},Y)$. One may verify that $\bm{\tilde{c}}$ also satisfies $\tilde{\bm{c}}\in\mathrm{SR}(\bm{\alpha},\bm{c})$, and so we conclude that the right-hand-side of Equation (12) contains $\mathsf{C}_{\mathsf{SAGE}}^{\mathsf{POLY}}(\bm{\alpha},X)$.

Now we address the reverse inclusion. Let $\bm{c}$ be such that $\mathrm{SR}(\bm{\alpha},\bm{c})\cap\mathsf{C}_{\mathsf{SAGE}}(\bm{\alpha},Y)$ is nonempty. One may verify that basic properties of $\mathsf{C}_{\mathsf{SAGE}}(\bm{\alpha},Y)$ and $\mathrm{SR}(\bm{\alpha},\bm{c})$ ensure that if the intersection is nonempty, it contains an element $\bm{\tilde{c}}$ satisfying $|\bm{c}|=|\bm{\tilde{c}}|$. Henceforth fix $\bm{\tilde{c}}$ satisfying these conditions. Next we appeal to a relaxed form of Corollary 5. Setting $\mathcal{N}=\{i:\tilde{c}_{i}\leq 0\}$, there exist vectors $\bm{\tilde{c}}^{(i)}$ satisfying

[TABLE]

Note how the definition of $\mathrm{SR}(\bm{\alpha},\bm{c})$ ensures that $\mathcal{N}=\{i:\bm{\alpha}_{i}\text{ is not even, or }c_{i}\leq 0\}$. Thus we define $\bm{c}^{(i)}$ by

[TABLE]

so that $\bm{c}=\sum_{i\in\mathcal{N}}\bm{c}^{(i)}$, and each $\bm{c}^{(i)}$ has the necessary sign pattern for membership in the $i^{\text{th}}$ AGE cone with respect to $\bm{\alpha},X$. Finally, note that

[TABLE]

to complete the proof. ∎

4.2 Solution recovery for sparse moment problems

This section concerns using dual SAGE relaxations to recover solutions to optimization problems of the form (1.1), where $f$ and $g_{i}$ are polynomials over exponents $\bm{\alpha}$. If $\bm{G}$ a $k\times m$ matrix whose $i^{\text{th}}$ row is the coefficient vector of $g_{i}$, $\bm{\alpha}_{1}$ is the zero vector, and $\bm{c}$ is the coefficient vector of $f$, then the simplest such relaxation is

[TABLE]

Overall, our goal is to recover vectors $\bm{x}$ in $X$ satisfying $\bm{v}=(\bm{x}^{\bm{\alpha}_{1}},\ldots,\bm{x}^{\bm{\alpha}_{m}})$, where $\bm{v}$ is an optimal solution to a relaxation such as the one above. We assume that $X$ is of a form where one of Theorems 9 or 10 provide a tractable representation of $\mathsf{C}_{\mathsf{SAGE}}(\bm{\alpha},X)$; this assumption allows us to leverage the following corollary.

Corollary 11**.**

Fix $Y=\{\bm{y}:H(\exp\bm{y})\leq\bm{1}\}$ for a continuous $H:\mathbb{R}^{n}\to\mathbb{R}^{r}$.

• •

If $X=\operatorname{cl}\{\bm{x}\,:\,\bm{0}<\bm{x},~{}H(\bm{x})\leq\bm{1}\}$, then $\mathsf{C}_{\mathsf{SAGE}}^{\mathsf{POLY}}(\bm{\alpha},X)^{\dagger}=\mathsf{C}_{\mathsf{SAGE}}(\bm{\alpha},Y)^{\dagger}$.

• •

If $X=\operatorname{cl}\{\bm{x}\,:\,\bm{0}<|\bm{x}|,~{}H(|\bm{x}|)\leq\bm{1}\}$, then

[TABLE]

We make a running assumption that “$Y$” is convex.

Solution recovery for polynomial optimization is more difficult than for signomial optimization, because monomials possess both signs and magnitudes. We propose a two-phase approach for this problem, where different techniques are used to recover variable magnitudes and variable signs. The main ideas for each phase are described in Sections 4.2.1 and 4.2.2, while the formal algorithms are given in the appendix. The recovered signs and magnitudes are then combined in an elementary way, as given by the following algorithm.

If $\bm{v}$ is optimal for Problem (13) and $\bm{v}=(\bm{x}^{\bm{\alpha}_{1}},\dots,\bm{x}^{\bm{\alpha}_{m}})$ for an elementwise nonzero $\bm{x}$ in $X$, then Algorithm 2 will return an optimal solution to Problem 1.1. As with Algorithm 1 in the signomial case, the authors find it useful to apply a simple local solver to the output of Algorithm 2 as a sort of solution refinement. We say “Algorithm 2L” in reference to such a method, with COBYLA as the local solver.

4.2.1 Recovering variable magnitudes

Given $\bm{v}$ in $\mathsf{C}_{\mathsf{SAGE}}^{\mathsf{POLY}}(\bm{\alpha},X)^{\dagger}$, we want to find $\bm{x}\in X$ satisfying $(\bm{x}^{\bm{\alpha}_{1}},\ldots,\bm{x}^{\bm{\alpha}_{m}})=|\bm{v}|$.

Regardless of whether $X$ is sign-symmetric or $X\subset\mathbb{R}^{n}_{+}$, the variable $\bm{v}\in\mathsf{C}_{\mathsf{SAGE}}^{\mathsf{POLY}}(\bm{\alpha},X)$ is associated with an auxiliary variable $\bm{\hat{v}}$ in $\mathsf{C}_{\mathsf{SAGE}}(\bm{\alpha},Y)$, and the variable $\bm{\hat{v}}$ is associated with additional auxiliary variables $\bm{z}_{i}$ as part of the dual $Y$-AGE signomial cones. As we discussed in Section 3.2, the vectors $\bm{y}_{i}=\bm{z}_{i}/\hat{v}_{i}$ belong to $Y$, and so the vectors $\bm{x}_{i}=\exp\bm{y}_{i}$ must belong to $X$. These vectors $\bm{x}_{i}$ are not only feasible with respect to $X$, but also satisfy

[TABLE]

under the binding-constraint and normalization conditions discussed in Section 3.2. Of course, if $\bm{\hat{v}}=|\bm{v}|$, then Equation (14) is precisely what we desire from our variable magnitudes. Since $\bm{\hat{v}}=|\bm{v}|$ always holds at least for $X\subset\mathbb{R}^{n}_{+}$, the vectors $\bm{x}_{i}=\exp(\bm{z}_{i}/\hat{v}_{i})$ are reasonable candidates for variable magnitudes.

When $X$ is sign-symmetric, it is possible that $\bm{\hat{v}}$ does not equal $|\bm{v}|$. This is particularly likely when $\bm{v}$ is subject to additional linear constraints, such as $\bm{G}\bm{v}\geq\bm{0}$. Therefore when $X$ is sign-symmetric it is worth considering variable magnitudes which supplement the ones described above. We propose that one picks a threshold $\epsilon_{0}>0$, computes

[TABLE]

and exponentiates $\bm{x}=\exp\bm{y}$. The role of $\epsilon_{0}$ is to ensure that $\bm{x}=\exp\bm{y}$ satisfies $|\bm{x}|^{\bm{\alpha}_{i}}\leq\epsilon_{0}$ whenever $v_{i}=0$. Extremely small values of $\epsilon_{0}$ (such as $10^{-100}$) would be reasonable here.

A formal statement of our method for magnitude recovery (Algorithm 3) can be found in the appendix.

4.2.2 Recovering variable signs

For $\bm{v}$ in $\mathbb{R}^{m}$, let $\bm{\alpha}^{-1}(\bm{v})$ denote the set of $\bm{x}\in\mathbb{R}^{n}$ satisfying $\bm{v}=(\bm{x}^{\bm{\alpha}_{1}},\ldots,\bm{x}^{\bm{\alpha}_{m}})$. Henceforth, fix $\bm{v}$ and assume $\bm{\alpha}^{-1}(\bm{v})$ is nonempty. Here we describe how to find vectors $\bm{s}$ in $\{+1,0,-1\}^{n}$ so that at least one $\bm{x}\in\bm{\alpha}^{-1}(\bm{v})$ satisfies $x_{i}>0$ when $s_{i}=+1$, $x_{i}=0$ when $s_{i}=0$, and $x_{i}<0$ when $s_{i}=-1$. Once we describe this process, we relax the problem slightly so that $s_{i}=+1$ allows $x_{i}=0$.

First we address when $s_{i}$ should equal zero. Let $U=\{i\in[m]\,:\,v_{i}\neq 0\}$. Consider how if some $\bm{x}\in\bm{\alpha}^{-1}(\bm{v})$ has $x_{j}=0$, then we must have $\alpha_{ij}=0$ for all $i$ in $U$ (else the equality $\bm{x}^{\bm{\alpha}_{i}}=v_{i}\neq 0$ would fail). Thus when $\alpha_{ij}=0$ for all $i$ in $U$, we set $s_{j}=0$ without loss of generality. Now let $W=\{j\in[n]\,:\,\alpha_{ij}>0\text{ for some }i\text{ in }U\}$; these are indices for which $s_{j}$ is not yet decided. Consider the vector $(\bm{v}<0)\in\{0,1\}^{n}$ with values $(\bm{v}<0)_{i}=1$ if $v_{i}<0$, and zero if otherwise. Let $\bm{\alpha}[U,:]$ be the submatrix of $\bm{\alpha}$ formed by rows $\{\bm{\alpha}_{i}\}_{i\in U}$, and similarly index $(\bm{v}<0)$. Finally, solve

[TABLE]

for $\bm{z}$ in $\{0,1\}^{n}$. The remaining $(s_{j})_{j\in W}$ are $s_{j}=-1$ if $z_{j}=1$ and $s_{j}=1$ otherwise.

An individual solution to (16) can be computed efficiently by Gaussian elimination over the finite field $\mathbb{F}_{2}$. Standard techniques for finite-field linear algebra also allow us to compute a basis for a null space of a matrix in mod 2 arithmetic (c.f. [Bard2009]), and so in principle one can readily recover all possible solutions to (16). In practice we must be careful, since the number of solutions to the linear system can easily be exponentially large in $n$ (for example, when $\bm{\alpha}\equiv\bm{0}_{n\times m}\mod 2$). Our formal algorithm for sign recovery accounts for this fact, and employs an additional hueristic to handle the case when (16) is inconsistent. See the appendix for details.

4.3 A first worked example

This section’s example is to minimize a function appearing in the formulation of the cyclic $n$-roots problem. The general cyclic $n$-roots problem is a challenging benchmark problem in computer algebra [PHCpack]. Our problem is to minimize

[TABLE]

over the box $X=[-1/2,1/2]^{7}$. To the authors’ knowledge, this problem was first used as an optimization benchmark in the work by Ray and Nataraj, on computing the extrema of polynomials over boxes [RN2008]. One may verify that $f_{X}^{\star}=-7$, and that this objective value is attained at $\bm{x}^{(1)}=\bm{1}/2$ and $\bm{x}^{(2)}=-\bm{1}/2$. Despite this problem’s simplicity, it requires nontrivial computational effort with SOS methods. The lowest relaxation order that allows Gloptipoly3 [gloptipoly] to compute $f_{X}^{\star}=-7$ results in a semidefinite program that takes MOSEK 90 seconds to solve with Machine $\bm{\mathsf{W}}$.

SAGE relaxations automatically exploit the structure in this problem. Since the seven functions $f_{i}(\bm{x})=1-64\prod_{j\neq i}x_{i}$ are $X$-AGE and sum to $f+7$, we have that $-7\leq f_{X}^{\mathrm{SAGE}}\leq f_{X}^{\star}$. To address the dual SAGE relaxation and solution recovery, we introduce the $8\times 7$ matrix $\bm{\alpha}$, with final row $\bm{\alpha}_{8}=\bm{0}$, $\alpha_{ii}=0$ for $i\leq 7$, and $\alpha_{ij}=1$ for the remaining entries. Next we write $X=\{\bm{x}:\bm{x}^{2}\leq\bm{1}/4\}$, and for $Y=\{\bm{y}:\exp(2\bm{y})\leq\bm{1}/4\}$ numerically solve

[TABLE]

MOSEK solves this problem in 0.01 seconds with Machine $\bm{\mathsf{W}}$.

We recover candidate magnitudes by using the eight $Y$-AGE cones associated with the auxiliary variable $\bm{\hat{v}}\in\mathsf{C}_{\mathsf{SAGE}}(\bm{\alpha},Y)^{\dagger}$. To machine precision, each of these AGE cones yields the same candidate magnitude $|\bm{x}|=\bm{1}/2$. The optimal moment vector $\bm{v}=\bm{1}/64$ is elementwise positive, and so sign-pattern recovery is a matter of finding all solutions to the system $\bm{\alpha}\bm{z}\equiv\bm{0}\mod 2$. There are exactly two solutions to this system: $\bm{z}^{(1)}=\bm{0}$, and $\bm{z}^{(2)}=\bm{1}$. The first of these gives rise to signs $\bm{s}^{(1)}=\bm{1}$, and the second of these results in $\bm{s}^{(2)}=-\bm{1}$. By combining these candidate signs with candidate magnitudes, we obtain candidate solutions $\{\bm{1}/2,-\bm{1}/2\}$; since these solutions are feasible and obtain objective values matching the SAGE bound, we conclude that both candidate solutions are minimizers of $f$ over $X$.

4.4 Reference hierarchies for POPs

If $X\subset\mathbb{R}^{n}_{+}$, then one should use the same hierarchies described in Section 3.4, where “$\operatorname{Sig}$” is replaced by “$\operatorname{Poly}$” and constraints that a function is “an $X$-SAGE signomial” are replaced by constraints that the function is “an $X$-SAGE polynomial.” This section focuses on the more complicated case when $X$ is sign-symmetric.

Our reference hierarchy for functionally constrained polynomial optimization is similar to that used for signomial programming. Let $f$, $\{g_{i}\}_{i=1}^{k_{1}}$, and $\{\phi_{i}\}_{i=1}^{k_{2}}$ be polynomials over common exponents $\bm{\alpha}\in\mathbb{N}^{m\times n}$, and fix sign-symmetric $X\subset\mathbb{R}^{n}$. Define $\bm{\hat{\alpha}}$ as the matrix formed by stacking $\bm{\alpha}$ on top of $2\bm{\alpha}$, and then removing any duplicate rows. The SAGE relaxation for $(f,g,\phi)_{X}^{\star}$ at level $(p,q,\ell)$ is then

[TABLE]

As before, the decision variables are $\gamma\in\mathbb{R}$, and the coefficient vectors of $\{s_{h}\}_{h\in g[q]}$, $\{z_{h}\}_{h\in\phi[q]}$. The main difference between (17) and it’s signomial equivalent (11), is that the Lagrange multipliers are slightly more complex in (17). This change was made to improve performance for problems where only a few rows of $\bm{\alpha}$ belong to the nonnegative even integer lattice.

Our minimax-free reference hierarchy for polynomial optimization is meaningfully different from the signomial case. We begin by assuming a representation $X=\operatorname{cl}\{\bm{x}\,:\,\bm{0}<|\bm{x}|,~{}H(|\bm{x}|)\leq\bm{1}\}$, and subsequently defining $Y=\{\bm{y}\,:\,H(\exp\bm{y})\leq\bm{1}\}$. Let $\mathcal{A}$ and $\mathcal{C}$ be operators on polynomials so that $f=\operatorname{Poly}(\mathcal{A}(f),\mathcal{C}(f))$ always holds, and let $\bm{s}$ be the vector in $\mathbb{R}^{m}$ with $s_{i}=1$ when $\bm{\alpha}_{i}$ is even, and $s_{i}=0$ otherwise. The SAGE relaxation for $f_{X}^{\star}$ at level $(p,q)$ is

[TABLE]

over optimization variables $\bm{c}$ and $\gamma$.

Formulation (18) uses two parameters out of desire to mitigate both sources of error in the SAGE polynomial cone: the error from replacing a polynomial by its signomial representative, and the error from replacing the signomial nonnegativity cone by the SAGE cone. As we show in Section 5.2, the signomial representative complexity parameter “$q$” can make the difference in our ability to solve problems when $X=\mathbb{R}^{n}$.

4.5 A second worked example

The following problem appears in work on “Bounded Degree Sums-of-Squares” (BSOS) and “Sparse Bounded Degree Sums-of-Squares” (Sparse-BSOS) methods for polynomial optimization [LTY2017, WLT2018]. The latter paper reports that BSOS and Sparse-BSOS compute $(f,g)_{\mathbb{R}^{6}}^{\star}=-0.41288$ in 44.5 and 82.1 seconds respectively, when using SDPT3-4.0 on a machine with a quad-core 2.6GHz Core i7 processor and 16GB RAM.

[TABLE]

This problem (Example 4) is a good example for conditional SAGE polynomial relaxations, because it allows for several choices in partial dualization.

The simplest choice is to use no partial dualization at all– simply solve relaxations of the form (17) with $X=\mathbb{R}^{n}$. Indeed, it is possible to solve Example 3 with only these ordinary SAGE certificates, however the necessary level of the hierarchy $(f,g)^{(1,1,0)}_{\mathbb{R}^{6}}=-0.41288$ requires 101 seconds of solver time on Machine $\bm{\mathsf{W}}$.

A strictly preferable alternative is to apply partial dualization with $X=\mathbb{R}^{6}_{+}$. With this choice of $X$ it is natural to drop the first two and last six constraints from $g$ (all of which will be trivially satisfied), and work with $\hat{g}=g_{3:10}$. This allows us to compute $(f,\hat{g})^{(1,1,0)}_{X}=-0.41288$ in 3.04 seconds of solver time on Machine $\bm{\mathsf{W}}$, and 4.4 seconds of solver time on Machine $\bm{\mathsf{L}}$. It is significant that the SAGE relaxation can be solved in this amount of time on Machine $\bm{\mathsf{L}}$, since it is an order of magnitude faster than BSOS solve time reported in [WLT2018].

The most aggressive choice for partial dualization is $X=\{\bm{x}\,:\,\bm{x}\geq\bm{0},g_{6:7}\geq\bm{0}\}$. With this choice of $X$ one has the option of using $\hat{g}=(g_{3:5},g_{8:10})$, or $\hat{g}=g_{3:10}$; in the first case Machine $\bm{\mathsf{W}}$ computes $(f,\hat{g})_{X}^{(1,1,0)}=-0.47121$ in 3.3 seconds, and in the second case Machine $\bm{\mathsf{W}}$ computes $(f,\hat{g})_{X}^{(1,1,0)}=-0.41288$ in 5.67 seconds. It is worth emphasizing that even though $g_{6:7}$ were incorporated into $X$, the SAGE bound with Lagrange multiplier complexity $p=1$ improved by including $g_{6:7}$ in the Lagrangian.

5 Computational experiments

This section presents the results of some computational experiments with SAGE relaxations. Experiments with signomial programs consist of twenty-nine problems drawn from the literature, of which seventeen are solved to optimality (see Section 5.1). Examples for polynomial optimization include twenty-two problems from the literature (Section 5.2), as well as randomly generated problems (Section 5.3).

All experiments described here were run with the provided sageopt python package. Sageopt is an extension and refinement of the “sigpy” package introduced by the authors in the appendix of [SAGE2]. In the spirit of Gloptipoly3 [gloptipoly], sageopt includes its own basic rewriting system to cast a SAGE relaxation into conic forms acceptable by appropriate solvers. The rewriting system also provides mechanisms for computing constraint violations, analyzing low-level problem data, and constructing a set $X$ (in an appropriate representation) from lists of constraint functions $g$ and $\phi$. Once problem data $f$, $g$, and $\phi$ are defined, a SAGE relaxation can be constructed and solved in just two lines of code. Solution recovery similarly requires no more than two lines of code. Sageopt currently supports the conic solvers ECOS [ecos, expConeThesis] and MOSEK [mosek].

Unless otherwise stated, experiments were conducted on Machine $\bm{\mathsf{W}}$. All experiments were conducted with the MOSEK solver, using the default solver tolerances. We note that although Sections 3.4 and 4.4 only stated the SAGE relaxations in primal form, these experiments were conducted by symbolically constructing primal and dual problems, and solving them separately from one another. In order to communicate the quality of these numeric solutions, we generally report “SAGE bounds” to the farthest decimal point where the primal and dual objectives agree.

5.1 Signomial programs from the literature

The examples in this section were drawn from the PhD thesis of James Yan [Yan1976], a popular benchmarking paper by Rijckaert and Martens [RM1978], and the more contemporary works [HSC2014, Xu2014]. The organization of this section is chronological with respect to these three sources. Many of the problems considered here can be found elsewhere in the literature, including work by Shen et al. [Shen2004, Shen2006, Shen2008], Wang and Liang [WL2005] and Qu et al. [Qu2007].

SAGE recovers best-known solutions for all but six of the twenty nine problems considered here. For every one of these six problematic examples, numerical issues resulted in solver failures for level-$(p,q,\ell)$ relaxations whenever $p>0$; the results for these six problems should not be taken as definitive. For the twenty-three problems where SAGE recovered best-known solutions, there are two important trends we can observe. First, our solution recovery algorithms are more likely to succeed with a conditional SAGE relaxation than with an ordinary SAGE relaxation, even when the ordinary SAGE relaxation is tight. Second– the local solver refinement in Algorithm 1L can help tremendously not only in the presence of suboptimal strictly-feasible initial solutions (Example 8), but also in the presence of both large and small constraint violations (Examples 9 and 6 respectively). The initial condition from a SAGE relaxation in Algorithm 1L is important; the underlying COBYLA solver can and will return suboptimal solutions if initialized poorly.

5.1.1 Problems from the PhD thesis of James Yan

We attempted to solve nine example problems appearing James Yan’s 1976 PhD thesis Signomial programs with equality constraints : numerical solution and applications [Yan1976]. This section reproduces two of the six problems which we solved to global optimality via SAGE certificates. Yan’s “Problem B” (page 88) and “Problem C” (page 89) serve as our Examples 5 and 6 respectively.

[TABLE]

It is possible to quickly compute $(f,g,\phi)_{\mathbb{R}^{4}}^{\star}=1.\overline{925}$ with both conditional and ordinary SAGE certificates, although conditional SAGE certificates exhibit better performance for solution recovery. Specifically, $(f,g,\phi)^{(1,1,0)}_{\mathbb{R}^{4}}=1.92592592$ can be computed in 0.12 seconds, but no solution can be recovered from Algorithm 1 unless $\epsilon$ is set to an unacceptably large value of $0.1$. Instead we set $X=\{\bm{x}\,:\,g(\bm{x})\geq\bm{0}\}$, compute $(f,g,\phi)_{X}^{(1,1,0)}=1.92592593$ in 0.18 seconds, and by running Algorithm 1 recover $\bm{x}^{\star}$ satisfying $g(\bm{x}^{\star})>$ 1E-11, $|\phi(\bm{x}^{\star})|<$ 1E-8, and $f(\bm{x}^{\star})=1.92592593$.

[TABLE]

Setting $X=\{\bm{x}\in\mathbb{R}^{3}\,:\,g(\bm{x})\geq\bm{0}\}$, we can compute $(f,g_{1:2},\phi)_{X}^{(0,1,0)}=-320.722913$ in 0.04 seconds. By running Algorithm 1 with $\epsilon_{\text{ineq}}=$1E-8 and $\epsilon_{\text{eq}}=$1E-6, we recover $\bm{x}$ with objective $f(\bm{x})=-320.722913$ and that is feasible up to tolerance 8E-7. We then pass this solution to COBYLA with parameter RHOEND=1E-10, and subsequently recover recover $\bm{x}^{\star}$ with the same objective, but constraint violation of only 5E-13.

The remaining problems which we solved to optimality were “Problem A” on page 60, “Problem A” on page 88, “Problem D” on page 89, and the problem in equation environment “(6.15)” on page 106. The last of these was introduced in Section 3.5 as “Example 2.” The problems which we did not solve to optimality were “Problem B” on page 61, the problem in equation environment “(6.29)” on page 113, and the problem in equation environment “(6.36)” on page 120. In each of these unsolved cases, we encountered solver-failures for level-$(p,q,\ell)$ relaxations whenever $p>0$. Therefore the bounds computed for each of these problems were essentially limited to those of Lagrange dual problems, with modest partial dualization.

5.1.2 Problems from the benchmarking paper of Rijckaert and Martens

We attempted to solve problems 9 through 18 of the popular signomial-geometric programming benchmark paper by Rijckaert and Martens [RM1978]. Of these ten problems, seven met with at least moderate success, in that SAGE relaxations produced meaningful lower bounds on a problem’s optimal value, and also facilitated recovery of best-known solutions to these problems. SAGE certificates allow us to certify global optimality for four of these seven problems. Problem statistics and a qualitative summary of SAGE performance is given in Table 2.

We reproduce Rijckaert and Martens’ problems 10 and 15 as our Examples 7 and 8 respectively; both problems are written in exponential-form.

[TABLE]

The bound constraints appearing in Example 7 are not included in [RM1978], however $f$ is unbounded below if we omit them. The solution proposed in [RM1978] has $\exp\bm{x}=(88.310,7.454,1.311)$, and objective value $f(\bm{x})=-83.06$. The actual optimal solution has value $-83.25$, and this can be certified by running Algorithm 1 on a dual solution for $f_{X}^{(3)}=-83.2510$, where $X=\{\bm{x}\,:\,g(\bm{x})\geq\bm{0}\}$. Solving the necessary SAGE relaxation takes 0.1 seconds on Machine $\bm{\mathsf{W}}$.

[TABLE]

A level (1,1,0) ordinary SAGE relaxation for Example 8 can be solved in 2.8 seconds on Machine $\bm{\mathsf{W}}$; this returns the bound $(f,g)_{\mathbb{R}^{10}}^{(1,1,0)}=0.2056534$. When Algorithm 1 is run on the dual solution, it returns a point $\bm{x}$ satisfying $f(\bm{x})\approx 0.38$ and $g(\bm{x})\geq 0.053$. However by subsequently running Algorithm 1L, we obtain $\bm{x}^{\star}$ satisfying $f(\bm{x}^{\star})=0.20565341$ and $g_{i}(\bm{x}^{\star})\geq$ 1E-8 for all $i$ in $[k]$. We thus conclude that the level-$(1,1,0)$ SAGE relaxation was tight.

5.1.3 Problems from contemporary sources

Here we describe our attempts at solving six problems from the 2014 article by Hou, Shen, and Chen [HSC2014], as well as four problems from the 2014 article by Xu [Xu2014]. SAGE relaxations are quite successful in this regard: seven of the ten problems are solved to global optimality (verified SAGE bounds), while best-known (but possibly suboptimal) solutions are obtained for the remaining three problems. Summary results can be found in Tables 4 and 5. We explicitly reproduce problem [HSC2014]-8 as our Example 9.

[TABLE]

Six of the fifteen variables in Example 9 have matching upper and lower bounds – these are the six equality constraints alluded to in Table 4. Our formulation differs from [HSC2014]-8, in that a constraint “$x_{3}x_{2}-x_{3}\leq 0$” in the original problem statement was replaced by “$y_{2}-y_{3}\leq 0$” in our problem statement. This change is necessary because the original problem is actually infeasible.

We approach Example 9 by maximizing our use of partial dualization: the set $X\subset\mathbb{R}^{15}$ includes all bound constraints, all but two of the first nine inequality constraints, as well as the constraint fourth from the end of the problem statement. The equality constraints implied for variables $y_{3},y_{4},y_{5},y_{6},y_{14},y_{15}$ are not included in the Lagrangian. A level-(0,1,0) conditional SAGE relaxation then produces a bound $(f,g)_{X}^{\star}\geq 156.2196$ in 0.05 seconds. By running Algorithm 1L with $\epsilon_{\text{ineq}}=100$, we subsequently obtain the geometric-form solution

[TABLE]

The solution $\bm{y}^{\star}$ is feasible up to forward-error 3.6E-14, and attains an objective value of $156.219629$. Because this objective matches the SAGE bound, we conclude that $\bm{y}^{\star}$ is optimal up to relative error 2E-7.

5.2 Polynomial optimization problems from the literature

Here we review results of the reference hierarchies from Section 4.4, as applied to twenty-two polynomial optimization problems from the literature. We begin with six unconstrained and eight box-constrained problems (drawn from [VLSE] and [RN2008] respectively). There are two important lessons which we highlight with the box-constrained problems. First, bound constraints should still be included in the Lagrangian, even if they can be completely absorbed into the set “$X$” in a conditional SAGE relaxation. Second, even if the original problem does not feature many sign-symmetric constraints, it is often easy to infer valid sign-symmetric constraints which can improve performance of a conditional SAGE relaxation. The remaining eight problems discussed in this section have nonconvex objectives, nonconvex inequality constraints, and constraints that the optimization variables are nonnegative [LTY2017]. Our experience with such problems is that partial dualization plays a crucial role in solving them efficiently, primarily with the simpler constraints $\bm{x}\geq 0$.

Table 6 gives problem data for the unconstrained and box-constrained problems; three such problems are reproduced here, as our Examples 10 through 12.

[TABLE]

The polynomial $f$ in Example 10 is known as the six-hump camel function; its minimum $f_{\mathbb{R}^{2}}^{\star}\approx-1.0316$ is attained at two points, which differ only by sign. By using polynomial modulators, a level-(3,0) relaxation returns a bound $-1.03170$ in 0.63 seconds of solver time on Machine $\bm{\mathsf{W}}$. By instead solving a level-(0,2) relaxation (i.e. modulating the signomial representative of $f-\gamma$) we obtain $-1.031630\leq f_{\mathbb{R}^{2}}^{\star}$ in 0.19 seconds. Example 10 shows how the two-parameter hierarchy in Section 4.4 can be of practical importance.

Our next two examples are box-constrained problems from the work of Ray and Nataraj [RN2008]; their problems “Capresse 4” and “Butcher 6” serve as our Examples 11 and 12. A consistent trend for these problems is that even when a feasible set $X$ can be incorporated entirely into an $X$-SAGE cone, it is still useful to take products of constraints, and solve a relaxation such as (17) which includes those constraints in the Lagrangian.

[TABLE]

Letting $X=\{\bm{x}\in\mathbb{R}^{4}\,:\,-0.5\leq x_{i}\leq 0.5\}$, one can compute $(f,g)_{X}^{(1,2,0)}=(f,g)_{\mathbb{R}^{4}}^{\star}=-3.1176903$, where the equality is verified by recovering a solution with Algorithm 2. Example 11 is noteworthy because the recovered solution required no local-solver refinement that occurs in Algorithm 2L.

[TABLE]

We can produce a tight bound for Example 12 with ordinary SAGE certificates: a level-(0,3,0) relaxation returns $-1.4392999\leq(f,g)^{\star}$ in 0.67 seconds. Solution recovery is not so easy. Unless we move to a computationally expensive level-(0,3,1) ordinary SAGE relaxation, Algorithm 2 fails to return a feasible point. Instead, we infer valid inequalities to describe a set “$X$” for use in a conditional SAGE relaxation:

[TABLE]

The resulting level (0,3,0) relaxation can be solved in 0.64 seconds. We recover a feasible solution with Algorithm 2, and obtain a solution matching the SAGE bound after refinement by COBYLA. Example 12 reinforces a message from signomial optimization: even if an ordinary SAGE relaxation can produce a tight bound, a conditional SAGE relaxation is likely to fare better with solution recovery. Example 12 also shows how useful sign-symmetric constraints can be inferred from a problem statement, even when those constraints are weaker than those found in the problem.

Now we turn to problems from [LTY2017], featuring nonconvex inequality constraints. One of these problems was introduced in Section 4.5 as “Example 4,” and all of these problems have a similar structure to that of Example 4. Most importantly, problems featured here include nonnegativity constraints $\bm{x}\geq\bm{0}$. The natural SAGE hierarchy produces tight bounds for all of these problems; results are summarized in Table 7.

There are a few subtle distinctions between geometric-form signomial programs (SPs), and nonnegative polynomial optimization problems (POPs) such as those considered here. While a polynomial optimization problem over $\bm{x}\geq\bm{0}$ may include $x_{i}=0$ in the feasible set, geometric-form SPs typically cannot allow this (since there is the possibility of dividing by zero, or encountering indeterminate forms). Thus solution recovery from SAGE relaxations is nominally more challenging for a true nonnegative POP, relative to a geometric-form SP. Despite this challenge, Algorithm 2L successfully recovers optimal solutions for all of these problems. See Table 8 for details.

The other important distinction between geometric-form SPs and nonnegative POPs, is that there exist established Sums-of-Squares based methods for dealing with nonnegative POPs. Thus it is useful to understand the performance of SAGE-based methods in the context of SOS-based methods for polynomial optimization. Although SAGE relaxations took a very long time to solve problems P4_6 and P4_8, the runtimes for problems such as P6_8 are remarkable. The unspecified machine in [LTY2017] took over 1600 and 200 seconds to solve P6_8 with BSOS and SOS respectively, while SAGE can solve the same problem in under 4 seconds on a mid-tier laptop from 2013. It seems to the authors that SAGE provides a compelling option for nonnegative polynomial optimization problems, at least for low levels of the hierarchy (such as $(1,1,0)$, or $(0,q,0)$ with small $q$).

5.3 Minimizing random sparse quartics over the sphere

In this section we describe how SAGE relaxations perform for minimizing sparse quartic forms over the unit sphere; this particular class of test problems is inspired from similar computational experiments by Ahmadi and Majumdar in their work on LP and SOCP-based inner-approximations of the SOS cone [SDSOS].

Our method for generating these problems is as follows: initialize $f=0$ as a polynomial in $n$ variables, and proceed to iterate over all tuples “$t$” in $[n]^{4}$. With probability $n\log n/n^{4}$, sample a coefficient $c_{t}$ from the standard normal distribution, and add the term $c_{t}\bm{x}^{\bm{\alpha}_{t}}$ to $f$, where $\bm{\alpha}_{t}\in[4]^{n}$ has $\alpha_{tj}=|\{i:t_{i}=j\}|$. The expected number of terms in $f$ after this procedure is roughly $n\log n$. Once a polynomial is generated, we solve a level-(0,2,0) conditional SAGE relaxation for $(f,g)^{\star}_{\mathbb{R}^{n}}$, where $g(\bm{x})=1-\bm{x}^{\intercal}\bm{x}$.555Because $f$ is homogeneous, $\bm{x}^{\intercal}\bm{x}=1$ may be relaxed to $\bm{x}^{\intercal}\bm{x}\leq 1$ without loss of generality The set “$X$” in the conditional SAGE relaxation is $X=\{\bm{x}:g(\bm{x})\geq 0\}$. Figure 1 and Table 9 report results for 20 problems in 10 variables, 20 problems in 20 variables, 14 problems in 30 variables, and 10 problems in 40 variables.

6 Discussion and Conclusion

In this article we introduced and developed notions of conditional SAGE certificates for both signomials and polynomials. In the signomial case, the underlying theory of the $X$-SAGE cones has deep roots in convex duality, while in the polynomial case, we derived efficient representations of important $X$-SAGE cones by employing “signomial representatives” introduced by the authors in earlier work. Through worked examples and computational experiments, we have demonstrated that subsequent convex relaxations can be used to solve many signomial and polynomial optimization problems from the literature. The authors believe that conditional SAGE certificates are a fertile area for research in both the theory and practice of constrained optimization; we briefly describe some possible directions here.

From an applications perspective, it would be interesting to see how conditional SAGE certificates can help with branch-and-bound algorithms. Whether considering signomials in geometric form, signomials in exponential form, or polynomials, bound constraints can easily be incorporated into $X$-SAGE cones. On the algorithmic front, important work remains to be done on solvers for relative entropy programs, primarily with regards to problems where the optimal solution contains a large number of variables along certain extreme rays of the exponential cone.

Conditional SAGE certificates raise many questions of potential interest to researchers in real algebraic geometry, and optimization-via-nonnegativity-certificates. Consider for example how the minimax-free hierarchy from Section 3.4 adopted a particular form for the modulating function: $\operatorname{Sig}(\bm{\alpha},\bm{1})^{\ell}$. What benefit might there be to instead using a modulator $\operatorname{Sig}(\bm{\hat{\alpha}},\bm{1})^{\ell}$, where $\bm{\hat{\alpha}}$ was chosen with consideration to $X$? Equally important, how could one efficiently identify good candidates for such $\bm{\hat{\alpha}}$, given only $\bm{\alpha}$ and a description of $X$? And at the most fundamental level, one asks – for what exponents $\bm{\alpha}$ and what sets $X$ do $X$-SAGE cones coincide with $X$-nonnegativity cones? Prior work (c.f. [SAGE2], and more recently [SONCboundary]) has uncovered meaningful sufficient conditions for this problem when $X=\mathbb{R}^{n}$. These sufficient conditions have hitherto been stated in terms of the combinatorial geometry of the exponent vectors $\{\bm{\alpha}_{i}\}_{i=1}^{m}$. It will be very interesting to see how such results do (or do not) generalize to $X$-SAGE cones for arbitrary $X$.

7 Appendix

As in the signomial case, Algorithm 3 always returns a vector $\bm{x}\in X$. Assuming that $\bm{z}$ from Line 7 are already computed as part of representing $\bm{\hat{v}}$, the complexity of this algorithm is dominated by Line 12. The runtime of Line 12 is in turn negligible relative to solving a SAGE relaxation to obtain vectors $\bm{v}$ and $\bm{\hat{v}}$. Infeasibility errors encountered in Line 12 should be handled by jumping to Line 15.

Let us describe the ways in which Algorithm 4 differs from the discussion in Section 4.2.2. First- there are changes to the sets $U$ and $W$. The set $U$ now drops any rows $\bm{\alpha}_{i}$ from $\bm{\alpha}$ where $\bm{\alpha}_{i}$ is even; it is easy to verify that this does not affect the set of solutions to the appropriate linear system. The set $W$ changes by only considering $j$ where at least one $\alpha_{ij}\equiv 1\mod 2$. This change is valid because if $\alpha_{ij}$ is even for all $i$, then the sign of variable $x_{j}$ is irrelevant to the underlying optimization problem, and we make take $x_{j}\geq 0$ without loss of generality.

Next we speak to the “hueristic” sign recovery. We partly mean to leave this as open-ended, however for completeness we describe the algorithm used in sageopt. The goal is to find a vector $\bm{s}$ in $\{+1,-1\}$ so that the signs of $\bm{s}^{\bm{\alpha}}\doteq(\bm{s}^{\bm{\alpha}_{1}},\ldots,\bm{s}^{\bm{\alpha}_{m}})$ match the signs of $\bm{v}$ to the greatest extent possible. However, we consider how having $\bm{s}^{\bm{\alpha}_{i}}$ match the sign of $v_{i}$ may not be very important if $v_{i}$ is very small. Therefore we use a merit function $M(\bm{s})=\bm{v}^{\intercal}\bm{s}^{\bm{\alpha}}$ to evaluate the quality of candidate signs $\bm{s}$. We apply a greedy algorithm to maximize the merit function $M(\bm{s})$ as follows: initialize $\bm{s}=\bm{1}$, and a set of undecided coordinates $C=\{1,\ldots,n\}$. As long as the set $C$ is nonempty, find an index $i^{\star}\in C$ so that changing $s_{i^{\star}}=1$ to $s_{i^{\star}}=-1$ maximizes improvement in the merit function. If the improvement is positive, then perform the update $s_{i^{\star}}\leftarrow-1$. Regardless of whether or not the improvement is positive, remove $i^{\star}$ from $C$. Once $C$ is empty, return $\bm{s}$.