Error bounds for monomial convexification in polynomial optimization

TL;DR

This paper provides explicit error bounds for convexifying monomials in polynomial optimization, enabling better understanding of approximation quality in convex relaxations.

Contribution

It introduces the first comprehensive error bounds for monomial convexification over subsets of [0,1]^n, including multilinear cases and convex hull derivations.

Findings

Error bounds depend mainly on monomial degree

Bounds are computationally straightforward to evaluate

Convex hull of multilinear monomials over [-1,1]^n derived

Abstract

Convex hulls of monomials have been widely studied in the literature, and monomial convexifications are implemented in global optimization software for relaxing polynomials. However, there has been no study of the error in the global optimum from such approaches. We give bounds on the worst-case error for convexifying a monomial over subsets of $[0,1]^n$. This implies additive error bounds for relaxing a polynomial optimization problem by convexifying each monomial separately. Our main error bounds depend primarily on the degree of the monomial, making them easy to compute. Since monomial convexification studies depend on the bounds on the associated variables, in the second part, we conduct an error analysis for a multilinear monomial over two different types of box constraints. As part of this analysis, we also derive the convex hull of a multilinear monomial over $[-1,1]^n$.

Equations

$$z_S^{\ast }=\min \{p(x)\mid x\in S\}$$

$$\mu \left(X\right):=\underset{(x,w)\in X}{\max }\left|w-x^{\alpha }\right|.$$

$$\mathcal{C}():=\left(1-\frac{1}{\left|\right|}\right){\left|\right|}^{\frac{1}{1-||}},{\mathcal{C}}_{||}^2:={\left(1-\frac{1}{\left|\right|}\right)}^{||}.$$

$$\mu \left(G({\mathrm{vex}}_S[\text{mathpzc}f])\right)\le {\left(1-\frac{1}{\left|\gamma \right|}\right)}^{|\gamma |}\le {\mathcal{C}}_{||}^2,\mu \left(G({\mathrm{cav}}_S[\text{mathpzc}f])\right)\le \mu \left(\mathrm{conv}{G}_S(\text{mathpzc}f)\right)\le \mathcal{C}(),$$

$${\gamma }_j:=\{\begin{array}{ll}\frac{1-(1-{\sigma }_j{)}^{{\alpha }_j}}{{\sigma }_j},& \text{if σj>0},\\ {\alpha }_j,& \text{if σj=0,}\end{array}j=1,\dots ,n.$$

$$L^{\prime }(p)=\max \left\{\underset{\alpha :c_{\alpha }>0}{\max }{c}_{\alpha }{\mathcal{C}}_{||}^2,\underset{\alpha :c_{\alpha }<0}{\max }-c_{\alpha }\mathcal{C}()\right\}.$$

$$z_S^{\ast }-z_S^{mono}\le L^{\prime }(p)\left(\genfrac{}{}{0ex}{}{n+m}{n}\right).$$

$$z_S^{\ast }-z_S^{mono}=\sum _{\alpha :c_{\alpha }>0}{c}_{\alpha }(x^{\alpha }-{\mathrm{vex}}_S[\text{mathpzc}f](x))+\sum _{\alpha :c_{\alpha }<0}(-c_{\alpha })({\mathrm{cav}}_S[\text{mathpzc}f](x)-x^{\alpha }).$$

$$z_S^{\ast }-z_S^{mono}\le \underset{\alpha }{\max }\left|c_{\alpha }\right|\left(1-\frac{1}{m}\right)m^{\frac{1}{1-m}}\left(\genfrac{}{}{0ex}{}{n+m}{n}\right).$$

$$\mu \left(G({\mathrm{cav}}_{{\Delta }_n}[\text{mathpzc}f])\right)\le \mu \left(\mathrm{conv}{G}_{{\Delta }_n}(\text{mathpzc}f)\right)\le \frac{({\alpha }^{\alpha }{)}^{1/||}}{\left|\right|}-\frac{{\alpha }^{\alpha }}{{\left|\right|}^{||}},\mu \left(G({\mathrm{vex}}_{{\Delta }_n}[\text{mathpzc}f])\right)=\frac{{\alpha }^{\alpha }}{{\left|\right|}^{||}}.$$

$$\begin{array}{r}{\mathcal{D}}_{r,n}:=\underset{i=1,\dots ,n-1}{\max }\left\{{\left(1+\frac{i}{n}(r-1)\right)}^n-r^i\right\},{\mathcal{E}}_{r,n}:=1+\frac{r^n-1}{(r-1)}\left[\frac{n-1}{n}{\left(\frac{r^n-1}{n(r-1)}\right)}^{\frac{1}{n-1}}-1\right].\end{array}$$

$$\mu\left(\operatorname{\mathcal{G}}(\operatorname{cav}_{[1,r]^{n}}[\mathpzc{m}])\right)=\mathscr{E}_{r,n},\quad\mu\left(\operatorname{\mathcal{G}}(\operatorname{vex}_{[1,r]^{n}}[\mathpzc{m}])\right)=\mathscr{D}_{r,n},\quad\mu\left(\operatorname{conv}$$

Full text

AMS subject classification. 90C26, 65G99, 52A27

1 Introduction

A polynomial $p\in\real[x]$, where $\real[x]=\real[x_{1},\dots,x_{n}]$ is the ring of $n$-variate polynomials, is a linear combination of monomials and is expressed as $p(x)=\sum_{\alpha}c_{\alpha}{x}^{\alpha}$ where the sum is finite, ${x}^{\alpha}\mathrel{\mathop{\ordinarycolon}}=\prod_{j=1}^{n}x_{j}^{\alpha_{j}}$ is a monomial, and every $\alpha_{j}$ is a nonnegative integer. A polynomial optimization problem is

[TABLE]

for a compact convex set $S$ and $p\in\real[x]$. It is common to assume that the degree of the polynomial is bounded by some constant $m$ and this is denoted by $p\in\real[x]_{m}$. Polynomials, in general, are nonconvex functions, thereby necessitating the use of global optimization algorithms for optimizing them. Strong and efficiently computable convex relaxations are a major component of these algorithms, making them a subject of ongoing research. One approach for devising good relaxations is based on taking the convex envelope of each polynomial $p(x)$ over $S$. However, since this computation is NP-hard even in the most basic cases having $m=2$ and $S=[0,1]^{n}$ or $S$ being a standard simplex, a main emphasis of the envelope studies has been on finding the envelope either under structural assumptions on $S$ or by considering only a subset of all the monomials appearing in $p(x)$. Also, one is interested in obtaining polyhedral relaxations of the envelope so that lower bounds can be computed cheaply by solving linear programs (LPs) iteratively [locatelli2014convex, meyer2005convex, tawarmalani2013explicit, sherali2012poly]. If $p(x)$ is a multilinear polynomial (i.e. $\alpha_{j}\in\{0,1\}$ for all $j$) and $S$ is a box, then the envelopes are polyhedral and we know exponential sized extended formulations [rikun1997convex, sherali1997convex], as well as valid inequalities [del2016polyhedral, crama2017class] and efficient cutting planes [misener2015dynamically, bao2015global] in projected spaces. A second method for obtaining lower bounds on the polynomial optimization problem has been to use the moments approach and [lasserre2001global] hierarchy of semidefinite relaxations (SDPs) that converges to the global optimum [lasserre2015introduction, laurent2009sums]. All of these techniques can of course also be used for relaxing a optimization problem that has polynomials in both the objective and constraints.

For a general polynomial $p(x)=\sum_{\alpha}c_{\alpha}{x}^{\alpha}$, given that it is hard to find the envelope explicitly and that computability of the SDP bounds does not scale well, a common relaxation technique, motivated by the classical work of [mccormick1976computability], has been to replace each monomial ${x}^{\alpha}$ with a continuous variable, say $w$, and then add inequalities to convexify the graph of ${x}^{\alpha}$ over $S$, which is the set $\{(x,w)\in S\times\real\mid w={x}^{\alpha}\}$. This is referred to as monomial convexification, and it typically yields a weaker relaxation than the envelope of the polynomial due to the fact that the envelope operator does not distribute over sums in general. However, because they may be cheaper and easier to generate than convexification of the entire polynomial, convex hulls of monomials have received significant attention [bao2015global, buchheim2016monomial, liberti2003convex, couenne] and are also routinely implemented in leading global optimization software [dalkiran2016rlt, misener2014antigone, tawarmalani2005polyhedral]. We still do not know an explicit form for the convex hull of a general monomial, but a number of results are available for bivariate monomials [locatelli2016polyhedral] and $n$-variate multilinear monomials [belotti2010valid, al1983jointly, benson2004concave, crama1993concave, mahdi2010coloring, meyer2004trilinear, ryoo2001analysis]. Moreover, there also exist challenging applications [buchheim2010integer] where the constraints can be formulated as having only monomial terms, thereby making monomial convexifications necessary for obtaining strong relaxations.

To quantify the strength of a relaxation of $p(x)$, one is interested in bounding the error produced with respect to the global optimum $z^{\ast}_{S}$ by optimizing over this relaxation. Error bounds for converging solutions of iterative optimization algorithms have been the subject of study before [pang1997error], but since these are not suited for studying relaxation strengths, different error measures have been proposed. [mahdi2010coloring] studied a relative error measure for the relaxation of a bilinear polynomial $p\in\real[x]_{2}$ over $S=[0,1]^{n}$ obtained by convexifying each monomial with its McCormick envelopes. They showed that for every $x\in[0,1]^{n}$, the ratio of the difference between the McCormick overestimator and underestimator values at $x$ and the difference between the concave and convex envelope values at $x$ can be bounded by a constant that is solely in terms of the chromatic number of the co-occurrence graph of the bilinear polynomial. Recently, [boland2017bounding] showed that this same ratio cannot be bounded by a constant independent of $n$. Another, and somewhat natural, way of measuring the error from a relaxation is to bound the absolute gap $z^{\ast}_{S}-\tilde{z}_{S}$, where $\tilde{z}_{S}$ is a lower bound on $z^{\ast}_{S}$ due to some convex relaxation of $\{(x,w)\in S\times\real\mid w=p(x)\}$. Such a bound helps determine how close one is to optimality in a global optimization algorithm. Also, there are examples (cf. $\prod_{j=1}^{n}{x}_{j}$ over $[1,r]^{n}$ in [mahdi2010coloring, pp. 332]) where the relative error gap of McCormick relaxation goes to $\infty$, while this can never happen with the absolute gap. The only result that we know of on bounding absolute gaps for general polynomials is due to [de2010error] who used Bernstein approximation of polynomials for a hierarchy of LP and SDP relaxations. (On the contrary, [de2016convergence, de2015error] bound the absolute error from upper bounds on $z^{\ast}_{S}$.). We mention that the absolute errors arising from piecewise linear relaxations of bilinear monomials appearing in a specific application were studied by [deygupte2013pooling]. Finally, a third error measure is based on comparing the volume of a convex relaxation to the volume of the convex hull. This has been done for McCormick relaxations of a trilinear monomial over a box by [speakman2015quantifying].

Our contribution.

In this paper, we bound the absolute gap to $z^{\ast}_{S}$ from monomial convexification and thereby add to the small number of explicit error bounds for polynomial optimization. To bound this gap, we analyze the error in relaxing a monomial with its convex hull. This error analysis not only implies a bound on the absolute gap to $z^{\ast}_{S}$ but it also can be used for bounding the error in relaxing any optimization problem with polynomials in both the objective and constraints. Our error measure is the maximum absolute deviation between the actual value and the approximate value of the monomial. Thus for any set $X$ in the $(x,w)$-space, we denote the error of $X$ with respect to ${x}^{\alpha}$ by $\mu\left(X\right)$, which is defined as

[TABLE]

We will mostly be interested in the error $\mu\left(\cdot\right)$ for the convex hull of the graph of ${x}^{\alpha}$ and for the convex and concave envelopes of ${x}^{\alpha}$. As mentioned earlier, monomial convexification errors have gone largely unnoticed in the literature, the only results being for the bilinear monomial $x_{1}x_{2}$. The folklore result [al1983jointly, cf.] for $x_{1}x_{2}$ over a rectangle $[l_{1},u_{1}]\times[l_{2},u_{2}]$ states that the convex hull and envelope errors are attained at $(x_{1},x_{2})=(\frac{u_{1}+l_{1}}{2},\frac{u_{2}+l_{2}}{2})$, which is the midpoint of the two diagonals of the box. [linderoth2005simplicial] derived error formulae for $x_{1}x_{2}$ over triangles created by the two diagonals of $[l_{1},u_{1}]\times[l_{2},u_{2}]$. Since convex hull and envelope results for a bilinear polynomial are invariant to affine transformations, it is equivalent to consider $x_{1}x_{2}$ over $[0,1]^{2}$. Substituting $n=2$ and $\alpha_{1}=\alpha_{2}=1$ in our forthcoming error bounds recover these known errors.

Notation.

The vector of ones is $\mathbf{e}$, the $i^{th}$ unit coordinate vector is $\mathbf{e}_{i}$, and the vector of zeros is $\mathbf{0}$; the dimensions will be apparent from the context in which these vectors are used. The convex hull of a set $X$ is $\operatorname{conv}X$ and the relative interior of $\operatorname{conv}X$ is $\operatorname{rel.int}X$. A nonempty box in n is $[l,u]\mathrel{\mathop{\ordinarycolon}}=[l_{1},u_{1}]\times\dots\times[l_{n},u_{n}]$. The standard boxes that we focus on in this paper are $[0,1]^{n},[-1,1]^{n}$, and $[1,r]^{n}$, for arbitrary scalar $r>1$. Another compact convex set of interest to us is the standard $n$-simplex $\Delta_{n}\mathrel{\mathop{\ordinarycolon}}=\operatorname{conv}\{\mathbf{0},\mathbf{e}_{1},\dots,\mathbf{e}_{n}\}=\left\{x\geq\mathbf{0}\mid\sum_{j=1}^{n}x_{j}\leq 1\right\}$. For convenience, we write $\mathpzc{f}(x)\mathrel{\mathop{\ordinarycolon}}={x}^{\alpha}$, $\mathpzc{f}^{\min}_{S}\mathrel{\mathop{\ordinarycolon}}=\min_{x\in S}\mathpzc{f}(x)$, $\mathpzc{f}^{\max}_{S}\mathrel{\mathop{\ordinarycolon}}=\max_{x\in S}\mathpzc{f}(x)$. The convex envelope of ${x}^{\alpha}$ over $S$, which is defined as the pointwise supremum of all convex underestimators of ${x}^{\alpha}$ over $S$, is denoted by $\operatorname{vex}_{S}[\mathpzc{f}]$. The concave envelope, which is analogously defined, is $\operatorname{cav}_{S}[\mathpzc{f}]$. The graph of a function $g(x)$ with domain $S$ is denoted by $\operatorname{\mathcal{G}}_{S}(g)\mathrel{\mathop{\ordinarycolon}}=\{(x,w)\in S\times\real\mid w=g(x)\}$. The graphs of the monomial and its envelopes are $\operatorname{\mathcal{G}}_{S}(\mathpzc{f})$, $\operatorname{\mathcal{G}}(\operatorname{vex}_{S}[\mathpzc{f}])$ and $\operatorname{\mathcal{G}}(\operatorname{cav}_{S}[\mathpzc{f}])$. Two special types of monomials are the symmetric monomial and the multilinear monomial. The former has $\alpha=\alpha_{0}\mathbf{e}$ for some $\alpha_{0}\in\mathbb{Z}_{\geq 1}$, and the latter, denoted by $\mathpzc{m}(x)\mathrel{\mathop{\ordinarycolon}}=\prod_{j=1}^{n}{x}_{j}$, is a special case of the former with $\alpha=\mathbf{e}$. For $\beta\in{}^{n}_{+}$, we denote $\mathinner{\!\left\lvert\beta\right\rvert}\mathrel{\mathop{\ordinarycolon}}=\sum_{j=1}^{n}\beta_{j}$.

1.1 Main results

We obtain strong and explicit upper bounds on $\mu\left(\cdot\right)$ for different types of monomials. In the polynomial optimization literature, it is common to assume, upto scaling and translation, that the domain $S$ of the problem is a subset of $[0,1]^{n}$. When analyzing a single monomial, this assumption is not without loss of generality since the monomial basis of $\real[x]$ is not closed upto translating and scaling the variables. Hence we divide our analysis into two parts. First, we consider a general monomial $\mathpzc{f}(x)={x}^{\alpha}$ over a compact convex set $S\subseteq[0,1]^{n}$, and bound the errors without using explicit analytic forms of the envelopes, which are hard to compute and unknown in closed form for arbitrary $S$. The concave error is bounded by computing the error from a specific concave overestimator that is precisely the concave envelope of ${x}^{\alpha}$ over $[0,1]^{n}$. On the convex side, we bound the error for any convex underestimator given as the pointwise supremum of (possibly uncountably many) linear functions, each of which underestimates ${x}^{\alpha}$ over $S$. Thus our error analysis has a distinctly polyhedral flavor.

In the second part, we limit our attention to a multilinear monomial $\mathpzc{m}(x)=\prod_{j=1}^{n}{x}_{j}$, but the domain $S$ is either a box with constant ratio or a symmetric box. By a box with constant ratio, we mean any box $[l,u]$ for which there exists a scalar $r>1$ such that $u_{i}/l_{i}=r$ for all $i$ with $l_{i}>0$, and $l_{i}/u_{i}=r$ for all $i$ with $l_{i}<0$. By a symmetric box, we mean any box $[l,u]$ that has $u_{i}=-l_{i}$ for all $i$. Since these boxes are simple scalings of $[1,r]^{n}$ and $[-1,1]^{n}$, respectively, and our error measure $\mu\left(\cdot\right)$ scales, we restrict our attention to only $[1,r]^{n}$ and $[-1,1]^{n}$. Contrary to the first part, here we first derive explicit polyhedral characterizations of the envelopes and convex hulls over $[1,r]^{n}$ and $[-1,1]^{n}$ and use them to perform a tight error analysis. The polyhedral representations for the $[1,r]^{n}$ case follow from the literature, whereas those over $[-1,1]^{n}$ are established in this paper.

1.1.1 General monomial

Consider a monomial ${x}^{\alpha}$ with $\alpha_{j}\in\mathbb{Z}_{\geq 1}$ for all $j$. The degree of this monomial is $\mathinner{\!\left\lvert\right\rvert}\mathrel{\mathop{\ordinarycolon}}=\mathinner{\!\left\lvert\alpha\right\rvert}=\sum_{j=1}^{n}\alpha_{j}$. The following constants will be useful throughout the paper:

[TABLE]

Theorem 1.1**.**

For the monomial $\mathpzc{f}(x)={x}^{\alpha}$ over $S\subseteq[0,1]^{n}$, we have

[TABLE]

where for $\sigma_{j}\mathrel{\mathop{\ordinarycolon}}=1-\max\{x_{j}\mid x\in S\}$, we define

[TABLE]

If $\mathbf{0},\mathbf{e}\in S$, then $\mu\left(\operatorname{conv}\operatorname{\mathcal{G}}_{S}(\mathpzc{f})\right)=\mu\left(\operatorname{\mathcal{G}}(\operatorname{cav}_{S}[\mathpzc{f}])\right)=\mathscr{C}()$.

The monotonicity of $\mathscr{C}()$ and $\mathscr{C}^{2}_{\mathinner{\!\left\lvert\right\rvert}}$ with respect to $\mathinner{\!\left\lvert\right\rvert}$ suggests the intuitive result that convexifying higher degree monomials will likely produce greater errors. As $\mathinner{\!\left\lvert\right\rvert}\to\infty$, we have $\mathscr{C}()\to 1$ and $\mathscr{C}^{2}_{\mathinner{\!\left\lvert\right\rvert}}\to 1/e$.

The bounds $\mathscr{C}()$ and $\mathscr{C}^{2}_{\mathinner{\!\left\lvert\right\rvert}}$ depend only on the degree of the monomial. They are a consequence of some general error bounds, established in Theorem LABEL:thm:concub for the concave error and in Theorem LABEL:thm:errenv01 for the convex error, that depend on how the monomial behaves over the domain $S$. The arguments used in proving Theorem 1.1 also imply that a family of convex relaxations of $\operatorname{\mathcal{G}}_{S}(\mathpzc{f})$ has error equal to $\mathscr{C}()$. We show this in Proposition LABEL:prop:Perr. We also guarantee in Corollary LABEL:corr:converr01multi that the convex envelope error bound $\mathscr{C}^{2}_{\mathinner{\!\left\lvert\right\rvert}}$ is tight for $\mathpzc{m}(x)$ over $S=[0,1]^{n}$.

Theorem 1.1 has two immediate implications. First, we obtain the error in convexifying a monomial over $[0,1]^{n}$.

Corollary 1.1**.**

$\mu\left(\operatorname{conv}\operatorname{\mathcal{G}}_{[0,1]^{n}}(\mathpzc{f})\right)=\mathscr{C}()$.

Second, we obtain an additive error bound on polynomial optimization over subsets of $[0,1]^{n}$. For a polynomial $p=\sum_{\alpha}c_{\alpha}{x}^{\alpha}\in\real[x]$, denote

[TABLE]

Let $z^{mono}_{S}\mathrel{\mathop{\ordinarycolon}}=\min\{\sum_{\alpha}c_{\alpha}w_{\alpha}\mid(x,w_{\alpha})\in\operatorname{conv}\operatorname{\mathcal{G}}_{S}(\mathpzc{f})\ \,\forall\alpha\}$ be the lower bound222To avoid tediousness and with a slight abuse of notation, for each monomial we write $(x,w_{\alpha})\in\operatorname{conv}\operatorname{\mathcal{G}}_{S}(\mathpzc{f})$ with the understanding that those $x_{j}$ that appear in the monomial are included. from monomial convexification on the global optimum $z^{\ast}_{S}=\min_{x\in S}p(x)$.

Corollary 1.2**.**

For any $p\in\real[x]_{m}$ and compact convex $S\subseteq[0,1]^{n}$,

[TABLE]

Proof.

We have $z^{mono}_{S}=\sum_{\alpha\colon c_{\alpha}>0}c_{\alpha}\operatorname{vex}_{S}[\mathpzc{f}](x)+\sum_{\alpha\colon c_{\alpha}<0}c_{\alpha}\operatorname{cav}_{S}[\mathpzc{f}](x)$. Therefore,

[TABLE]

Applying Theorem 1.1 and the construction of $L^{\prime}(p)$ gives us $z^{\ast}_{S}-z^{mono}_{S}\leq L^{\prime}(p)\sum_{\alpha}1$. Since $p\in\real[x]_{m}$, there are at most $\binom{n+m}{n}$ monomials in $p(x)$, leading to the claimed error bound. ∎

Computing $L^{\prime}(p)$ may get tedious if $p(x)$ has a large number of monomials. A cheaper bound is possible by considering only the largest coefficient in $p(x)$.

Corollary 1.3**.**

For any $p\in\real[x]_{m}$ and compact convex $S\subseteq[0,1]^{n}$,

[TABLE]

Proof.

Follows from Corollary 1.2 after using $\mathinner{\!\left\lvert\right\rvert}\leq m$ and $\mathscr{C}()$ being monotone in $\mathinner{\!\left\lvert\right\rvert}$. ∎

The bounds from Theorem 1.1, although applicable to arbitrary $S\subseteq[0,1]^{n}$, can be weak if $\mathbf{0}\in S$ and $\mathbf{e}\notin S$. To emphasize this, we consider a monomial over the standard simplex $\Delta_{n}$ and obtain error bounds that depend on not just the degree of the monomial but also the exponent of each variable. These bounds are stronger than the bounds $\mathscr{C}()$ and $\mathscr{C}^{2}_{\mathinner{\!\left\lvert\right\rvert}}$.

Theorem 1.2**.**

[TABLE]

All of the above bounds are tight for a symmetric monomial.

1.1.2 Multilinear monomial

Consider the multilinear monomial $\mathpzc{m}(x)=\prod_{j=1}^{n}{x}_{j}$.

Theorem 1.3**.**

Denote

[TABLE]

For $\mathpzc{m}(x)$ over $[1,r]^{n}$,

[TABLE]