Parametric Convex Quadratic Relaxation of the Quadratic Knapsac Problem

TL;DR

This paper introduces a parametric convex quadratic relaxation for the quadratic knapsack problem, optimizing perturbations and valid inequalities to obtain tighter upper bounds, with theoretical analysis and practical algorithms.

Contribution

It proposes a novel parametric quadratic relaxation method with a primal-dual interior point approach and valid inequalities for improved bounds on QKP.

Findings

The relaxation provides tighter upper bounds than existing methods.

The perturbation approach cannot improve bounds when applied to SDP relaxations.

Valid inequalities from cover and knapsack problems enhance the relaxation's effectiveness.

Abstract

We consider a parametric convex quadratic programming, CQP, relaxation for the quadratic knapsack problem, QKP. This relaxation maintains partial quadratic information from the original QKP by perturbing the objective function to obtain a concave quadratic term. The nonconcave part generated by the perturbation is then linearized by a standard approach that lifts the problem to the matrix space. We present a primal-dual interior point method to optimize the perturbation of the quadratic function, in a search for the tightest upper bound for the \QKPp. We prove that the same perturbation approach, when applied in the context of semidefinite programming, SDP, relaxations of the QKP, cannot improve the upper bound given by the corresponding linear \SDP relaxation. The result also applies to more general integer quadratic problems. Finally, we propose new valid inequalities on the lifted matrix variable, derived from cover and knapsack inequalities for the QKP, and present the separation problems to generate cuts for the current solution of the CQP relaxation. Our best bounds are obtained from alternating between optimizing the parametric quadratic relaxation over the perturbation and adding cutting planes generated by the valid inequalities proposed.

Tables

Table 1. Table 1: Equations number corresponding to acronyms

QKP	(1)	CI	(23)
QKP ${}_{\text{lifted}}$	(2)	ECI	(24)
LPR	(3)	LCI	(25)
CQP ${}_{Q_p}$	(5)	SCI	(28)
LSDP	(14)	CILS	(30)
QSDP ${}_{Q_p}$	(15)	SCILS	(32)
		SKILS	(37)

Table 2. Table 2: List of abbreviations

CQP	Convex Quadratic Programming
QKP	Quadratic Knapsack Problem
SDP	Semidefinte Programming
MIQP	Mixed Integer Quadratic Programming
MILP	Mixed Integer Linear Programming

Table 3. Table 3: Impact of the cuts added to LPR ~ ~ LPR \tilde{\mbox{LPR}} on 10 small instances ( n = 10 𝑛 10 n=10 ).

Method	OptGap	Time	Iter	Cuts	Time${}_{\text{MIP}}$
	(%)	(sec)			(sec)
$\widetilde{\text{LPR}}$	38.082	0.35	1.0
SCI1	36.703	32.38	1.1	28.4
SCI	10.036	39.98	3.0	364.1
CILS	19.719	9.00	2.7	82.2	6.91
SCILS	9.121	266.81	50.0	794.3	198.12
ALL	3.315	315.82	28.3	646.6	264.91

Table 4. Table 4: SDP bound vs IPM bound at iterations 1 and 20, for two initial matrices Q p 0 superscript subscript 𝑄 𝑝 0 Q_{p}^{0} .

$Q_p^0$	Inst	$n$	$ga{p}_1$ (%)	$ga{p}_2$ (%)
$Q_p^a$	I1	50	0.30	0.01
	I2	50	0.73	0.03
	I3	50	0.14	0.00
	I4	50	1.02	0.21
	I5	50	0.59	0.09
	I1	100	1.47	0.14
	I2	100	0.59	0.04
	I3	100	0.51	0.05
	I4	100	1.38	0.26
	I5	100	0.73	0.06
$Q_p^b$	I1	50	0.01	0.00
	I2	50	0.30	0.09
	I3	50	0.08	0.03
	I4	50	0.10	0.02
	I5	50	0.03	0.03
	I1	100	0.04	0.00
	I2	100	0.02	0.01
	I3	100	0.03	0.01
	I4	100	0.12	0.04
	I5	100	0.02	0.01

Table 5. Table 5: Results for CWICS ( n = 50 𝑛 50 n=50 ).

Inst	OptGap	Time	DuGap	Iter	Time${}_{\text{MIP}}$
	(%)	(sec)	(%)		(sec)
I1	0.23	1013.50	0.27	100	641.98
I2	0.00	632.50	0.00	64	411.67
I3	0.00	392.55	0.00	44	205.70
I4	0.00	289.97	0.00	31	160.37
I5	0.21	1093.60	0.37	10	698.04

Table 6. Table 6: Results for CWICS ( n = 100 𝑛 100 n=100 ).

Inst	OptGap	Time	DuGap	Iter	Time${}_{\text{MIP}}$
	(%)	(sec)	(%)		(sec)
I1	0.00	2035.30	0.00	20	737.86
I2	0.25	2177.30	0.65	20	919.41
I3	0.00	2007.10	0.00	20	773.00
I4	0.12	1885.90	0.84	20	828.98
I5	0.04	2309.50	0.20	20	970.49

Equations

$$\text{({QKP}\,)}\qquad\qquad\begin{array}[]{rrl}p^{*}_{\textsc{\tiny QKP}}:=&\max&x^{T}Qx\\ &\text{s.t.}&w^{T}x\leq c\\ &&x\in\{0,1\}^{n},\end{array}$$

$$\mathrm{svec}(A):=(a_{11},\dots ,a_{n1},a_{22},\dots ,a_{n2},\dots ,a_{nn}{)}^T\in {\mathbb{R}}^{n(n+1)/2}.$$

$$\{(x,X):w^{T}x\le c,X=x{x}^T,x\in \{0,1{\}}^n\}\subset \mathcal{P}.$$

$$Q-Q_p⪯0,$$

$$x^{T}Qx$$

$$=x^T(Q-Q_p)x+\mathrm{trace}(Q_{p}X).$$

$$(\textbf{CQP}\,_{Q_{p}})\qquad\qquad\begin{array}[]{rll}\textit{$p^{*}_{\textsc{\tiny CQP}}(Q_{p})$}:=\max&x^{T}(Q-Q_{p})x+\operatorname{{trace}}(Q_{p}X)\\ \mbox{s.t.}&(x,X)\in\mathcal{P}.\end{array}$$

$$para{m}_{\text{textsc}QKP}^{\ast }:=\underset{Q-Q_p⪯0}{\min }{p}_{\text{textsc}CQP}^{\ast }(Q_p).$$

$$\begin{array}[]{rll}\min&B_{\mu}(Q_{p},Z):=p^{*}_{\textsc{\tiny CQP}}(Q_{p})-\mu\log\det Z\\ \text{s.t.}&Q-Q_{p}+Z=0&(:\Lambda)\\ &Z\succ 0,&\end{array}$$

$$L_{\mu }(Q_p,Z,\Lambda ):=p_{\text{textsc}CQP}^{\ast }(Q_p)-\mu \log \det Z+\mathrm{trace}((Q-Q_p+Z)\Lambda ).$$

$$x^T(Q-Q_p)x+\mathrm{trace}(Q_{p}X)=x^{T}Qx+\mathrm{trace}(Q_p(X-x{x}^T)),$$

$$D(p_{\text{textsc}CQP}^{\ast }(Q_p);\Delta Q_p)=\underset{\text{optimal }x,X}{\max }\mathrm{trace}((X-x{x}^T)\Delta Q_p).$$

$$\nabla p_{\text{textsc}CQP}^{\ast }(Q_p)=X-x{x}^T.$$

$$\nabla B_{\mu }(Q_p)=(X-x{x}^T)-\mu Z^{-1}.$$

$$\begin{array}[]{lrcll}\frac{\partial L_{\mu}}{\partial Q_{p}}:&\nabla p^{*}_{\textsc{\tiny CQP}}(Q_{p})-\Lambda&=&0,\\ \frac{\partial L_{\mu}}{\partial\Lambda}:&Q-Q_{p}+Z&=&0,\\ \frac{\partial L_{\mu}}{\partial Z}:&-\mu Z^{-1}+\Lambda&=&0,&(\text{or})\,\,Z\Lambda-\mu I=0.\end{array}$$

$$\text{Gμ(Qp,Λ,Z)}=\left(\begin{array}{c}\nabla p_{\text{textsc}CQP}^{\ast }(Q_p)-\Lambda \\ Q-Q_p+Z\\ Z\Lambda -\mu I\end{array}\right)=0,Z,\Lambda \succ 0.$$

$$Y_k:=\nabla p_{\text{textsc}CQP}^{\ast }(Q_p^{k+1})-\nabla p_{\text{textsc}CQP}^{\ast }(Q_p^k),S_k:=Q_p^{k+1}-Q_p^k,$$

$$\upsilon :=⟨Y_k,S_k⟩,\omega :=⟨\mathrm{svec}(S_k),B_k\mathrm{svec}(S_k)⟩.$$

$$B_{k+1}:=B_k+\frac{1}{\upsilon }\left(\mathrm{svec}(Y_k)\mathrm{svec}(Y_k^T)\right)-\frac{1}{\omega }\left(B_k\mathrm{svec}(S_k)\mathrm{svec}(S_k{)}^T{B}_k\right).$$

$$\text{Gμ′(Qp,Λ,Z)}\left(\begin{array}{c}\Delta Q_p\\ \Delta \Lambda \\ \Delta Z\end{array}\right)=-G_{\mu }(Q_p,\Lambda ,Z),$$

$$G_{\mu }(Q_p,\Lambda ,Z)=\left(\begin{array}{c}\nabla p_{\text{textsc}CQP}^{\ast }(Q_p)-\Lambda \\ Q-Q_p+Z\\ Z\Lambda -\mu I\end{array}\right)=:\left(\begin{array}{c}{R}_d\\ R_p\\ R_c\end{array}\right).$$

$$\left\{\begin{array}[]{l}\operatorname{{sMat}}(B\operatorname{{svec}}(\Delta Q_{p}))-\Delta\Lambda=-R_{d},\\ -\Delta Q_{p}+\Delta Z=-R_{p},\\ Z\Delta\Lambda+\Delta Z\Lambda=-R_{c}.\end{array}\right.$$

$$\begin{array}[]{c}\Delta\Lambda=\operatorname{{sMat}}(B\operatorname{{svec}}(\Delta Q_{p}))+R_{d},\quad\Delta Z=-R_{p}+\Delta Q_{p}.\\ \end{array}$$

$$Z\mathrm{sMat}(B\mathrm{svec}(\Delta Q_p))+(\Delta Q_p)\Lambda =-R_c-Z{R}_d+R_p\Lambda .$$

$$\left(\begin{array}{c}{R}_d\\ R_p\\ R_c\end{array}\right):=\left(\begin{array}{c}\nabla p_{\text{textsc}CQP}^{\ast }(Q_p^k)-{\Lambda }^k\\ Q-Q_p^k+Z^k\\ Z^k{\Lambda }^k-{\mu }^{k}I\end{array}\right).$$

$$Z^k\mathrm{sMat}(B_k\mathrm{svec}(\Delta Q_p))+(\Delta Q_p){\Lambda }^k=-R_c-Z^k{R}_d+R_p{\Lambda }^k.$$

$$\Delta \Lambda :=\mathrm{sMat}(B_k\mathrm{svec}(\Delta Q_p))+R_d,\Delta Z:=-R_p+\Delta Q_p.$$

$$Q_p^{k+1}:=Q_p^k+{\hat{\alpha }}_p\Delta Q_p,Z^{k+1}:=Z_p^k+{\hat{\alpha }}_p\Delta Z,{\Lambda }^{k+1}:={\Lambda }^k+{\hat{\alpha }}_d\Delta \Lambda ,$$

$${\hat{\alpha }}_p:={\tau }_{\alpha }\times \min \{1,\text{mbox}{argmax}_{{\alpha }_p}\{Z_p^k+{\alpha }_p\Delta Z⪰0\}\},$$

$${\hat{\alpha }}_d:={\tau }_{\alpha }\times \min \{1,\text{mbox}{argmax}_{{\alpha }_d}\{{\Lambda }^k+{\alpha }_d\Delta \Lambda ⪰0\}\}.$$

$$\nabla p_{\text{textsc}CQP}^{\ast }(Q_p^{k+1}):=X(Q_p^{k+1})-x(Q_p^{k+1})x(Q_p^{k+1}{)}^T.$$

Full text

††journal: European Journal of Operational ResearchCorrCorrfootnotetext: Corresponding author: Marcia Fampa, PESC/COPPE, Universidade Federal do Rio de Janeiro, Caixa postal 68511, Rio de Janeiro, RJ, 21941-972, Brazil. Email: fampa@cos.ufrj.br

1 Introduction

We study a convex quadratic programming (CQP) relaxation of the quadratic knapsack problem (QKP),

[TABLE]

where $Q\in\mathbb{S}^{n}$ is a symmetric $n\times n$ nonnegative integer profit matrix, $w\in\mathbb{Z}^{n}_{++}$ is a vector of positive integer weights for the items, and $c\in\mathbb{Z}_{++}$ is the knapsack capacity with $c\geq w_{i},$ for all $i\in N:=\{1,\ldots,n\}$. The binary (vector) variable $x$ indicates which items are chosen for the knapsack, and the inequality in the model, known as a knapsack inequality, ensures that the selection of items does not exceed the knapsack capacity. We note that any linear costs in the objective can be included on the diagonal of $Q$ by exploiting the $\{0,1\}$ constraints and, therefore, are not considered.

The QKP was introduced in [12] and was proved to be NP-Hard in the strong sense by reduction from the clique problem. The quadratic knapsack problem is a generalization of the knapsack problem, which has the same feasible set of the QKP, and a linear objective function in $x$. The linear knapsack problem can be solved in pseudo-polynomial time using dynamic programming approaches with complexity of $O(nc)$.

The QKP appears in a wide variety of fields, such as biology, logistics, capital budgeting, telecommunications and graph theory, and has received a lot of attention in the last decades. Several papers have proposed branch-and-bound algorithms for the QKP, and the main difference between them is the method used to obtain upper bounds for the subproblems [7, 5, 6, 15, 16]. The well known trade-off between the strength of the bounds and the computational effort required to obtain them is intensively discussed in [24], where semidefinite programming (SDP) relaxations proposed in [15] and [16] are presented as the strongest relaxations for the QKP. The linear programming (LP) relaxation proposed in [5], on the other side, is presented as the most computationally inexpensive.

Both the SDP and the LP relaxations have a common feature, they are defined in the symmetric matrix lifted space determined by the equation $X=xx^{T}$, and by the replacement of the quadratic objective function in QKP with a linear function in $X$, namely, $\operatorname{{trace}}(QX)$. As the constraint $X=xx^{T}$ is nonconvex, it is relaxed by convex constraints in the relaxations. The well known McCormick inequalities [21], and also the semidefinite constraint, $X-xx^{T}\succeq 0$, have been extensively used to relax the nonconvex constraint $X=xx^{T}$, in relaxations of the QKP.

In this paper, we investigate a CQP relaxation for the QKP, where instead of linearizing the objective function, we perturb the objective function Hessian $Q$, and maintain the (concave) perturbed version of the quadratic function in the objective, linearizing only the remaining part derived from the perturbation. Our relaxation is a parametric convex quadratic problem, defined as a function of a matrix parameter $Q_{p}$, such that $Q-Q_{p}\preceq 0$. This matrix parameter is iteratively optimized by a primal-dual interior point method (IPM) to generate the best possible bound for the QKP. During this iterative procedure, valid cuts are added to the formulation to strengthen the relaxation, and the search for the best perturbation is adapted accordingly. Our procedure alternates between optimizing the matrix parameter and applying cutting planes generated by valid inequalities. At each iteration of the procedure, a new bound for the QKP is computed, considering the updated matrix parameter and the cuts already added to the relaxation.

A similar approach to handle nonconvex quadratic functions consists in decomposing it as a difference of convex (DC) quadratic function [18]. DC decompositions have been extensively used in the literature to generate convex quadratic relaxations of nonconvex quadratic problems. See, for example, [10] and references therein. Unlike the approach used in DC decompositions, we do not necessarily decompose $x^{T}Qx$ as a difference of convex functions, or equivalently, as a sum of a convex and a concave function. Instead, we decompose it as a sum of a concave function and a quadratic term derived from the perturbation applied to $Q$. This perturbation can be any symmetric matrix $Q_{p}$, such that $Q-Q_{p}\preceq 0$.

In an attempt to obtain stronger bounds, we also investigated the parametric convex quadratic SDP problem, where we add to our CQP relaxation, the positive semidefinite constraint $X-xx^{T}\succeq 0$. An IPM could also be applied to this parametric problem in order to generate the best possible bound. Nevertheless, we prove an interesting result concerning the relaxations, in case the constraint $X-xx^{T}\succeq 0$ is imposed: the tightest bound generated by the parametric quadratic SDP relaxation is obtained when the perturbation $Q_{p}$ is equal to $Q$, or equivalently, when we linearize the entire objective function, obtaining the standard linear SDP relaxation. We conclude, therefore, that keeping the (concave) perturbed version of the quadratic function in the objective of the SDP relaxation does not lead to a tighter bound.

Another contribution of this work is the development of valid inequalities for the CQP relaxation on the lifted matrix variable. The inequalities are first derived from cover inequalities for the knapsack problem. The idea is then extended to knapsack inequalities. Taking advantage of the lifting $X:=xx^{T}$, we propose new valid inequalities that can also be applied to more general relaxations of binary quadratic programming problems that use the same lifting. We discuss how cuts for the quadratic relaxation can be obtained by the solution of separation problems, and investigate possible dominance relation between the inequalities proposed.

Finally, we present our algorithm CWICS (Convexification With Integrated Cut Strengthening), where we iteratively improve the upper bound for the QKP by optimizing the choice of the perturbation of the objective function and adding cutting planes to the relaxation. At each iteration, lower bounds for the problem are also generated from feasible solutions constructed from a rank-one approximation of the solution of the CQP relaxation.

In Section 2, we introduce our parametric convex quadratic relaxation for the QKP. In Section 3, we explain how we optimize the parametric problem over the perturbation of the objective; i.e., we present the IPM applied to obtain the perturbation that leads to the best possible bound. In Section 4, we present our conclusion about the parametric quadratic SDP relaxation. In Section 5, we introduce new valid inequalities on the lifted matrix variable of the convex quadratic model, and we describe how cutting planes are obtained by the solution of separation problems. In Section 6, we present the heuristic used to generate lower bounds to the QKP. In Section 7, we present our algorithm CWICS, and discuss our numerical experiments, and in Section 8, we present our final remarks.

Notation

If $A\in\mathbb{S}^{n}$, then $\operatorname{{svec}}(A)$ is a vector whose entries come from $A$ by stacking up its ‘lower half’, i.e.,

[TABLE]

The operator $\operatorname{{sMat}}$ is the inverse of $\operatorname{{svec}}$, i.e., $\operatorname{{sMat}}(\operatorname{{svec}}(A))=A$.

We also denote by $\lambda_{\min}(A)$, the smallest eigenvalue of $A$ and by $\lambda_{i}(A)$ the $i^{th}$ largest eigenvalue of $A$.

To facilitate the reading of the paper, Table 1 relates the acronyms used with the associated equations numbers.

We also show the standard abbreviations used in the paper in Table 2.

2 A Parametric Convex Quadratic Relaxation

In order to construct a convex relaxation for QKP, we start by considering the following standard reformulation of the problem in the lifted space of symmetric matrices, defined by the lifting $X:=xx^{T}$.

[TABLE]

We consider an initial LP relaxation of QKP, given by

[TABLE]

where $\mathcal{P}\subset[0,1]^{n}\times\mathbb{S}^{n}$ is a bounded polyhedron, such that

[TABLE]

2.1 The perturbation of the quadratic objective

Next, we propose a convex quadratic relaxation with the same feasible set as LPR  , but maintaining a concave perturbed version of the quadratic objective function of QKP, and linearizing only the remaining nonconcave part derived from the perturbation. More specifically, we choose $Q_{p}\in\mathbb{S}^{n}$ such that

[TABLE]

and we get

[TABLE]

Finally, we define the parametric convex quadratic relaxation of QKP :

[TABLE]

3 Optimizing the parametric problem over the parameter $Q_{p}$

The upper bound $p^{*}_{\textsc{\tiny CQP}}(Q_{p})$ in the convex quadratic problem CQPQp depends on the feasible perturbation $Q_{p}$ of the Hessian $Q$. To find the best upper bound, we consider the parametric problem

[TABLE]

We solve (6) with a primal-dual interior-point method (IPM), and we describe in this section how the search direction of the algorithm is obtained at each iteration.

We start with minimizing a log-barrier function. We use the barrier function, $B_{\mu}(Q_{p},Z)$ with barrier parameter, $\mu>0$, to obtain the barrier problem

[TABLE]

where $Z\in\mathbb{S}^{n}$ and $\Lambda\in\mathbb{S}^{n}$ denote, respectively, the slack and the dual symmetric matrix variables. We consider the Lagrangian function

[TABLE]

Some important points should be emphasized here. We first note that the objective function for $p^{*}_{\textsc{\tiny CQP}}(Q_{p})$ is linear in $Q_{p}$, i.e., this function is the maximum of linear functions over feasible points $x,X$. Therefore, this is a convex function.

Moreover, as will be detailed next, the search direction of the IPM, computed at each iteration of the algorithm, depends on the optimum solution $x=x(Q_{p}),X=X(Q_{p})$ of CQPQp, for a fixed matrix $Q_{p}$. At each iteration of the IPM, we have $Z\succ 0$, and therefore $Q-Q_{p}\prec 0$. Thus, problem CQPQp maximizes a strictly concave quadratic function, subject to linear constraints over a compact set $\mathcal{P}$, and consequently, has a unique optimal solution (see e.g. [25]). From standard sensitivity analysis results, e.g.  [11, Corollary 3.4.2],[17], [9, Theorem 1], as the optimal solution $x=x(Q_{p}),X=X(Q_{p})$ is unique, the function $p^{*}_{\textsc{\tiny CQP}}(Q_{p})$ is differentiable and the gradient is obtained by differentiating the Lagrangian function.

Since $Q_{p}$ appears only in the objective function in CQPQp, and

[TABLE]

we get a directional derivative at $Q_{p}$ in the direction $\Delta Q_{p}$,

[TABLE]

Since we have a unique optimum $x=x(Q_{p}),X=X(Q_{p})$, we get the gradient

[TABLE]

The gradient of the barrier function, is then

[TABLE]

The optimality conditions for (7) are obtained by differentiating the Lagrangian $L_{\mu}$ with respect to $Q_{p},\Lambda,Z$, respectively,

[TABLE]

This gives rise to the nonlinear system

[TABLE]

We use a BFGS approximation for the Hessian of $p^{*}_{\textsc{\tiny CQP}}$, since it is not guaranteed to be twice differentiable everywhere, and update it at each iteration (see [20]). We denote the approximation of $\nabla^{2}_{\textsc{\tiny BFGS}}p^{*}_{\textsc{\tiny CQP}}(Q_{p})$ by $B$, and begin with the approximation $B_{0}=I$. Recall that if $Q_{p}^{k},Q_{p}^{k+1}$ are two successive iterates with gradients $\nabla p^{*}_{\textsc{\tiny CQP}}(Q^{k}_{p}),\nabla p^{*}_{\textsc{\tiny CQP}}(Q^{k+1}_{p})$, respectively, with current Hessian approximation $B_{k}\in\mathbb{S}^{n(n+1)/2}$, then we set

[TABLE]

and,

[TABLE]

Finally, we update the Hessian approximation with

[TABLE]

We note that the curvature condition $\upsilon>0$ should be verified to guarantee the positive definiteness of the updated Hessian. In our implementation, we address this by skipping the BFGS update when $\upsilon$ is negative or too close to zero.

The equation for the search direction is

[TABLE]

where

[TABLE]

If $B$ is the current estimate of the Hessian, then the system becomes

[TABLE]

We can substitute for the variables $\Delta\Lambda$ and $\Delta Z$ in the third equation of the system. The elimination gives us a simplified system, and therefore, we apply it, using the following two equations for elimination and backsolving,

[TABLE]

Accordingly, we have a single equation to solve, and the system finally becomes

[TABLE]

We emphasize that to compute the search direction at each iteration of our IPM, we need to update the residuals defined in (12), and therefore we need the optimal solution $x=x(Q_{p}),X=X(Q_{p})$ of the convex quadratic relaxation CQPQp for the current perturbation $Q_{p}$. Problem CQPQp is thus solved at each iteration of the IPM method, each time for a new perturbation $Q_{p}$, such that $Q-Q_{p}\prec 0$.

In Algorithm 1, we present in details an iteration of the IPM. The algorithm is part of the complete framework used to generate bounds for QKP, as described in Section 7.

Remark 1**.**

Algorithm is an interior-point method with a quasi-Newton step (BFGS). The object function we are minimizing is differentiable with exception possibly at the optimum. A complete convergence analysis of the algorithm is not in the scope of this paper, however, convergence analysis for some similar problems can be found in the literature. In [1], it is shown that if the objective function is always differentiable and strongly convex, then it is globally convergent to the analytic center of the primal-dual optimal set when $\mu$ tends to zero and strict complementarity holds.

4 The parametric quadratic SDP relaxation

In an attempt to obtain tighter bounds, a promising approach might seem to be to include the positive semidefinite constraint $X-xx^{T}\succeq 0$ in our parametric quadratic relaxation CQPQp, and solve a parametric convex quadratic SDP relaxation, also using an IPM. Nevertheless, we show in this section that the convex quadratic SDP relaxation cannot generate a better bound than the linear SDP relaxation, obtained when we set $Q_{p}$ equal to $Q$. In fact, as shown below, the result applies not only to the QKP, but to more general problems as well. We emphasize here that the same result does not apply for CQPQp. We could observe with our computational experiments that the best bounds were obtained by CQPQp, when we had $Q-Q_{p}\neq 0$, for all instances considered.

Consider the linear SDP problem given by

[TABLE]

where $x\in\mathbb{R}^{n}$, $X\in\mathbb{S}^{n}$, and $\mathcal{F}$ is any subset of $\mathbb{R}^{n}\times\mathbb{S}^{n}$.

We now consider the parametric SDP problem given by

[TABLE]

where $Q-Q_{p}\preceq 0$.

Theorem 2**.**

Let $\mathcal{F}$ be any subset of $\mathbb{R}^{n}\times\mathbb{S}^{n}$. For any choice of matrix $Q_{p}$ satisfying $Q-Q_{p}\preceq 0$, we have

[TABLE]

Moreover, \inf\{p^{*}_{\textsc{\tiny QSDP{}{Q{p}}}}~{}:~{}Q-Q_{p}\preceq 0\}=p^{*}_{\textsc{\tiny LSDP}}.

Proof.

Let $(\tilde{x},\tilde{X})$ be a feasible solution for LSDP . We have

[TABLE]

The inequality (17) holds because $(\tilde{x},\tilde{X})$ is also a feasible solution for QSDP ${}_{Q_{p}}$. The inequality in (20) holds because of the negative semidefiniteness of $Q-Q_{p}$ and $\tilde{x}\tilde{x}^{T}-\tilde{X}$. Because p^{*}_{\textsc{\tiny QSDP{}{Q{p}}}} is an upper bound on the objective value of LSDP  at any feasible solution, we can conclude that p^{*}_{\textsc{\tiny QSDP{}{Q{p}}}}\geq p^{*}_{\textsc{\tiny LSDP}}. Clearly, $Q_{p}=Q$ satisfies $Q-Q_{p}=0\preceq 0$ and LSDP  is the same as QSDP ${}_{Q_{p}}$ for this choice of $Q_{p}$. Therefore, \inf\{p^{*}_{\textsc{\tiny QSDP{}{Q{p}}}}~{}:~{}Q-Q_{p}\preceq 0\}=p^{*}_{\textsc{\tiny LSDP}}. ∎

Notice that in Theorem 2 we do not require that $\mathcal{F}$ be convex nor bounded. Also, in principle, for some choices of $Q_{p}$, we could have p^{*}_{\textsc{\tiny QSDP{}{Q{p}}}}=+\infty with $p^{*}_{\textsc{\tiny LSDP}}=+\infty$ or not.

Remark 3**.**

As a corollary from Theorem 2, we have that the upper bound for QKP, given by the solution of the quadratic relaxation CQPQp, cannot be smaller than the upper bound given by the solution of the SDP relaxation obtained from it, by adding the SDP constraint $X-xx^{T}\succeq 0$ and setting $Q_{p}$ equal to $Q$.

5 Valid inequalities

We are now interested in finding valid inequalities to strengthen relaxations of QKP in the lifted space determined by the lifting $X:=xx^{T}$. Let us denote by CRel, any convex relaxation of QKP in the lifted space, where the equation $X=xx^{T}$ was relaxed in some manner, by convex constraints, i.e., any convex relaxation of QKP ${}_{\mbox{lifted}}$

We note that if the inequality

[TABLE]

is valid for QKP, where $\tau\in\mathbb{Z}_{+}^{n}$ and $\beta\in\mathbb{Z}_{+}$, then, as $x$ is nonnegative and $X:=xx^{T}$,

[TABLE]

is a valid inequality for QKP ${}_{\mbox{lifted}}$. In this case, we say that (22) is a valid inequality for QKP ${}_{\mbox{lifted}}$ derived from the valid inequality (21) for QKP.

5.1 Preliminaries: knapsack polytope and cover inequalities

We begin by recall the concepts of knapsack polytopes and cover inequalities.

The knapsack polytope is the convex hull of the feasible points of the knapsack problem,

[TABLE]

Definition 4** (zero-one knapsack polytope).**

[TABLE]

Proposition 5**.**

The dimension

[TABLE]

and KPol  is an independence system, i.e.,

[TABLE]

Proof.

Recall that $w_{i}\leq c,\forall i$. Therefore, all the unit vectors $e_{i}\in\mathbb{R}^{n}$, as well as the zero vector, are feasible, and the first statement follows. The second statement is clear. ∎

Cover inequalities were originally presented in [3, 26]; see also [23, Section II.2]. These inequalities can be used in general optimization problems with knapsack inequalities and binary variables and, particularly, in QKP.

Definition 6** (cover inequality, CI).**

The subset $C\subseteq N$ is a cover if it satisfies

[TABLE]

The (valid) CI is

[TABLE]

The cover inequality is minimal if no proper subset of $C$ is also a cover.

Definition 7** (extended CI, ECI).**

Let $w^{*}:=\max_{j\in C}w_{j}$ and define the extension of $C$ as

[TABLE]

The ECI is

[TABLE]

Definition 8** (lifted CI, LCI).**

Given a cover $C$, let $\alpha_{j}\geq 0,\forall j\in N\backslash C$, and $\alpha_{j}>0$, for some $j\in N\backslash C$, such that

[TABLE]

is a valid inequality for KPol . Inequality (25) is a LCI .

Cover inequalities are extensively discussed in [14, 4, 3, 26, 23, 2]. Details about the computational complexity of LCI  is presented in [28, 13]. Algorithm 2 [27, page 2], shows how to derive a facet-defining LCI  from a given minimal cover $C$.

5.2 Adding cuts to the relaxation

Given a solution $(\bar{x},\bar{X})$ of CRel, our initial goal is to obtain a valid inequality for QKP ${}_{\mbox{lifted}}$ derived from a CI  that is violated by $(\bar{x},\bar{X})$. A CI  is formulated as $\alpha^{T}x\leq e^{T}\alpha-1$, where $\alpha\in\{0,1\}^{n}$ and $e$ denotes the vector of ones. We then search for the CI  that maximizes the sum of the violations among the inequalities in $\bar{Y}\textbf{cut}(\alpha)\leq 0$, where $\bar{Y}:=\left(\bar{x}\;\bar{X}\right)$ and

[TABLE]

To obtain such a CI , we solve the following linear knapsack problem,

[TABLE]

Let $\alpha^{*}$ solve (27). If $v^{*}>0$, then at least one valid inequality in the following set of $n$ scaled cover inequalities, denoted in the following by SCI, is violated by $(\bar{x},\bar{X})$.

[TABLE]

Based on the following theorem, we note that to strengthen cut (28), we may apply Algorithm 2 to the CI   obtained, lifting it to an LCI , and finally add the valid inequality (22) derived from the LCI  to CRel.

Theorem 9**.**

The valid inequality (22) for QKP ${}_{\mbox{lifted}}$, which is derived from a valid LCI , dominates all inequalities derived from a CI  that can be lifted to the LCI .

Proof.

Consider the LCI  (25) derived from a CI  (23) for QKP. The corresponding scaled cover inequalities (22) derived from the CI  and the LCI  are, respectively,

[TABLE]

and

[TABLE]

where $\alpha_{j}\geq 0,\forall j\in N\backslash C$. Clearly, as all $X_{ij}$ are nonnegative, the second inequality dominates the first, for all $i\in N$. ∎

5.3 New valid inequalities in the lifted space

As discussed, after finding any valid inequality in the form of (21) for QKP, we may add the constraint (22) to CRel when aiming at better bounds. We observe now, that besides (22) we can also generate other valid inequalities in the lifted space by taking advantage of the lifting $X:=xx^{T}$, and also of the fact that $x$ is binary. In the following, we show how the idea can be applied to cover inequalities.

Let

[TABLE]

where $C\subset N$ and $\beta<|C|$, be a valid inequality for KPol .

Inequality (29) can be either a cover inequality, CI , an extended cover inequality, ECI , or a particular lifted cover inequality, LCI , where $\alpha_{j}\in\{0,1\},\forall j\in N\backslash C$ in (25). Furthermore, given a general LCI , where $\alpha_{j}\in\mathbb{Z}_{+}$, for all $j\in N\backslash C$, a valid inequality of type (29) can be constructed by replacing each $\alpha_{j}$ with $\min\{\alpha_{j},1\}$ in the LCI .

Definition 10** (Cover inequality in the lifted space, CILS ).**

Let $C\subset N$ and $\beta<|C|$ as in inequality (29), and also consider here that $\beta>1$. We define

[TABLE]

as the CILS derived from (29).

Theorem 11**.**

If inequality (29) is valid for QKP, then the CILS    (30) is a valid inequality for QKP ${}_{\mbox{lifted}}$.

Proof.

Considering (29), we conclude that at most $\binom{\beta}{2}$ products of variables $x_{i}x_{j}$, where $i,j\in C$, can be equal to 1. Therefore, as $X_{ij}:=x_{i}x_{j}$, the result follows. ∎

Remark 12**.**

When $\beta>1$, inequality (29) is well known as a clique cut, widely used to model decision problems, and frequently used as a cut in branch-and-cut algorithms. In this case, using similar idea to what was used to construct the CILS, we conclude that it possible to fix

[TABLE]

Given a solution $(\bar{x},\bar{X})$ of CRel, the following MIQP problem is a separation problem, which searches for a CILS  violated by $\bar{X}$.

[TABLE]

If $\alpha^{*},\beta^{*},K^{*}$ solves MIQP 1, with $z^{*}>0$, the CILS  given by $\operatorname{{trace}}(K^{*}X)\leq\beta^{*}(\beta^{*}-1)$ is violated by $\bar{X}$. The binary vector $\alpha^{*}$ defines the CI  from which the cut is derived. The CI  is specifically given by ${\alpha^{*}}^{T}x\leq e^{T}{\alpha^{*}}-1$ and $\beta^{*}(\beta^{*}-1)$ determines the right-hand side of the CILS. The inequality is multiplied by 2 because we consider the variable $K$ as a symmetric matrix, in order to simplify the presentation of the model.

Theorem 13**.**

The valid inequality CILS  for QKP ${}_{\mbox{lifted}}$, which is derived from a valid LCI  in the form (29), dominates any CILS  derived from a CI  that can be lifted to the LCI.

Proof.

As $X$ is nonnegative, it is straightforward to verify that if $X$ satisfies a CILS  derived from a LCI, $X$ also satisfies any CILS  derived from a CI  that can be lifted to the LCI. ∎

Any feasible solution of MIQP 1 such that $\operatorname{{trace}}(\bar{X}K)>\beta(\beta-1)$ generates a valid inequality for QKP ${}_{\mbox{lifted}}$ that is violated by $\bar{X}$. Therefore, we do not need to solve MIQP 1 to optimality to generate a cut. Moreover, to generate distinct cuts, we can solve MIQP 1 several times (not necessarily to optimality), each time adding to it, the following “no-good” cut to avoid the previously generated cuts:

[TABLE]

where $\bar{\alpha}$ is the value of the variable $\alpha$ in the solution of MIQP 1 when generating the previous cut.

We note that, if $\alpha^{*},\beta^{*},K^{*}$ solves MIQP 1, then ${\alpha^{*}}^{\prime}x\leq e^{\prime}\alpha^{*}-1$ is a valid CI  for QKP, however it may not be a minimal cover. Aiming at generating stronger valid cuts, based in Theorem 13, we might add to the objective function of MIQP 1, the term $-\delta e^{\prime}\alpha$, for some weight $\delta>0$. The objective function would then favor minimal covers, which could be lifted to a facet-defining LCI, that would finally generate the CILS. We should also emphasize that if the CILS  derived from a CI  is violated by a given $\bar{X}$, then clearly, the CILS  derived from the LCI  will also be violated by $\bar{X}$.

Now, we also note that, besides defining one cover inequality in the lifted space considering all possible pairs of indexes in $C$, we can also define a set of cover inequalities in the lifted space, considering for each inequality, a partition of the indexes in $C$ into subsets of cardinality 1 or 2. In this case, the right-hand side of the inequalities is never greater than $\beta/2$. The idea is made precise below.

Definition 14** (Set of cover inequalities in the lifted space, SCILS).**

Let $C\subset N$ and $\beta<|C|$ as in inequality (29). Let

$C_{s}:=\{(i_{1},j_{1}),\ldots,(i_{p},j_{p})\}$ be a partition of $C$, if $|C|$ is even. 2. 2.

$C_{s}:=\{(i_{1},j_{1}),\ldots,(i_{p},j_{p})\}$ be a partition of $C\setminus\{i_{0}\}\,\,\mbox{for each}\,\,i_{0}\in C$, if $|C|$ is odd and $\beta$ is odd. 3. 3.

$C_{s}:=\{(i_{0},i_{0}),(i_{1},j_{1}),\ldots,(i_{p},j_{p})\}$, where $\{(i_{1},j_{1}),\ldots,(i_{p},j_{p})\}$ is a partition of $C\setminus\{i_{0}\}\,\,\mbox{for each}\,\,i_{0}\in C$, if $|C|$ is odd and $\beta$ is even.

In all cases, $i_{k}<j_{k}$ for all $k=1,\ldots,p$.

The inequalities in the SCILS derived from (29) are given by

[TABLE]

for all partitions $C_{s}$ defined as above.

Theorem 15**.**

If inequality (29) is valid for QKP, then the inequalities in the SCILS    (32) are valid for QKP ${}_{\mbox{lifted}}$.

Proof.

The proof of the validity of SCILS  is based on the lifting relation $X_{ij}=x_{i}x_{j}$. We note that if the binary variable $x_{i}$ indicates whether or not the item $i$ is selected in the solution, the variable $X_{ij}$ indicates whether or not the pair of items $i$ and $j$, are both selected in the solution.

If $|C|$ is even, $C_{s}$ is a partition of $C$ in exactly $|C|/2$ subsets with two elements each, and therefore, if at most $\beta$ elements of $C$ can be selected in the solution, clearly at most $\left\lfloor\frac{\beta}{2}\right\rfloor$ subsets of $C_{s}$ can also be selected. 2. 2.

If $|C|$ and $\beta$ are odd, $C_{s}$ is a partition of $C\setminus\{i_{0}\}$ in exactly $|C-1|/2$ subsets with two elements each, where $i_{0}$ can be any element of $C$. In this case, if at most $\beta$ elements of $C$ can be selected in the solution, clearly at most $\frac{\beta-1}{2}\left(=\left\lfloor\frac{\beta}{2}\right\rfloor\right)$ subsets of $C_{s}$ can also be selected. 3. 3.

If $|C|$ is odd and $\beta$ is even, $C_{s}$ is the union of $\{(i_{0},i_{0})\}$ with a partition of $C\setminus\{i_{0}\}$ in exactly $|C-1|/2$ subsets with two elements each, where $i_{0}$ can be any element of $C$. In this case, if at most $\beta$ elements of $C$ can be selected in the solution, clearly at most $\frac{\beta}{2}\left(=\left\lfloor\frac{\beta}{2}\right\rfloor\right)$ subsets of $C_{s}$ can also be selected.

∎

Given a solution $(\bar{x},\bar{X})$ of CRel, we now present a mixed linear integer programming (MILP) separation problem, which searches for an inequality in SCILS  that is most violated by $\bar{X}$. Let $A\in\{0,1\}^{n\times\frac{n(n+1)}{2}}$. In the first $n$ columns of $A$ we have the $n\times n$ identity matrix. In the remaining $n(n-1)/2$ columns of the matrix, there are exactly two elements equal to 1 in each column. All columns are distinct. For example, for $n=4$,

[TABLE]

The columns of $A$ represent all the subsets of items in $N$ with one or two elements. Let

[TABLE]

If $\alpha^{*},v^{*},K^{*},y^{*}$ solves MILP 2, with $z^{*}>0$, then the particular inequality in SCILS  given by

[TABLE]

is violated by $\bar{X}$. The binary vector $\alpha^{*}$ defines the CI  from which the cut is derived. As the CI  is given by $\alpha^{*}x\leq e^{\prime}\alpha^{*}-1$, we can conclude that the cut generated either belongs to case (1) or (3) in Definition 14. This fact is considered in the formulation of MILP 2. The vector $y^{*}$ defines a partition $C_{s}$ as presented in case (3), if $\sum_{i=1}^{n}y(i)=1$, and in case (1), otherwise. We finally note that the number 2 in the right-hand side of (33) is due to the symmetry of the matrix $K^{*}$.

We now may repeat the observations made for MIQP 1.

Any feasible solution of MILP 2 such that $\operatorname{{trace}}(\bar{X}K)>2v$ generates a valid inequality for CRel , which is violated by $\bar{X}$. Therefore, we do not need to solve MILP 2 to optimality to generate a cut. Moreover, to generate distinct cuts, we can solve MILP 2 several times (not necessarily to optimality), each time adding to it, the following suitable “no-good” cut to avoid the previously generated cuts:

[TABLE]

where $\bar{y}$ is the value of the variable $y$ in the solution of MILP 2, when generating the previous cut.

The CI   ${\alpha^{*}}^{\prime}x\leq e^{\prime}\alpha^{*}-1$ may not be a minimal cover. Aiming at generating stronger valid cuts, we might add again to the objective function of MILP 2, the term $-\delta e^{\prime}\alpha$, for some weight $\delta>0$. The objective function would then favor minimal covers, which could be lifted to a facet-defining LCI. In this case, however, after computing the LCI, we have to solve MILP 2 again, with $\alpha$ fixed at values that represent the LCI, and $v$ fixed so that the right-hand side of the inequality is equal to the right-hand side of the LCI. All components of $y$ that were equal to 1 in the previous solution of MILP 2 should also be fixed at 1. The new solution of MILP 2 would indicate the other subsets of $N$ to be added to $C_{s}$. One last detail should be taken into account. If the cover $C$ corresponding to the LCI , is such that $|C|$ is odd and the right-hand side of the LCI  is also odd, then the cut generated should belong to case (2) in Definition 14, and MILP 2 should be modified accordingly. Specifically, the second and third constraints in MILP 2, should be modified respectively to

[TABLE]

Remark 16**.**

Let $\gamma:=|C|$. Then, the number of inequalities in the SCILS  is

[TABLE]

if $\gamma$ is even, or

[TABLE]

if $\gamma$ is odd.

Finally, we extend the ideas presented above to the more general case of knapsack inequalities. We note that the following discussion applies to a general LCI , where $\alpha_{j}\in\mathbb{Z}_{+},\forall j\in N\backslash C$.

Let

[TABLE]

be a valid knapsack inequality for KPol , with $\alpha_{j},\beta\in\mathbb{Z}_{+},\beta\geq\alpha_{j},\forall j\in N$.

Definition 17** (Set of knapsack inequalities in the lifted space, SKILS).**

Let $\alpha_{j}$ be the coefficient of $x_{j}$ in (35). Let $\{C_{1},\ldots,C_{q}\}$ be the partition of $N$, such that $\alpha_{u}=\alpha_{v}$, if $u,v\in C_{k}$ for some $k$, and $\alpha_{u}\neq\alpha_{v}$, otherwise. The knapsack inequality (35) can then be rewritten as

[TABLE]

Now, for $k=1,\dots,q$, let $C_{l_{k}}:=\{(i_{k_{1}},j_{k_{1}}),\ldots,(i_{k_{p_{k}}},j_{k_{p_{k}}})\}$, where $i<j$ for all $(i,j)\in C_{l_{k}}$, and

$C_{l_{k}}$ is a partition of $C_{k}$, if $|C_{k}|$ is even.

• 2.

$C_{l_{k}}$ is a partition of $C_{k}\setminus\{i_{k_{0}}\}$, where $i_{k_{0}}\in C_{k}$, if $|C_{k}|$ is odd.

The inequalities in the SKILS corresponding to (35) are given by

[TABLE]

for all partitions $C_{l_{k}}$, $k=1,\ldots,q$, defined as above, and for all $i_{k_{0}}\in C_{k}\setminus C_{l_{k}}$. (If $|C_{k}|$ is even, $C_{k}\setminus C_{l_{k}}=\emptyset$, and the term in the variable $X_{i_{k_{0}}i_{k_{0}}}$ does not exist.)

Remark 18**.**

Consider $\{C_{1},\ldots,C_{q}\}$ as in Definition 17. For $k=1,\ldots,q$, let $\gamma_{k}:=|C_{k}|$ and define

[TABLE]

if $\gamma_{k}$ is even, or

[TABLE]

if $\gamma_{k}$ is odd.

Then, the number of inequalities in SKILS  is

[TABLE]

Remark 19**.**

If $\tilde{\alpha}_{k}$ is even for every $k$, such that $\gamma_{k}$ is odd, then the right-hand side $\beta$ of inequality (37) may be replaced with $2\times\left\lfloor\frac{\beta}{2}\right\rfloor$, which will strengthen the inequality in case $\beta$ is odd.

Note that the case where $\gamma_{k}:=|C_{k}|$ is even for every $k$, is a particular case contemplated by this remark, where the the tightness of the inequality can also be applied.

Corollary 20**.**

If inequality (35) is valid for QKP , then the inequalities (37), in the SKILS  ,  are valid for QKP ${}_{\mbox{lifted}}$, whether or not the modification suggested in Remark 19 is applied.

Proof.

The result is again verified, by using the same argument used in the proof of Theorem 15, i.e., considering that $X_{ij}=1$, iff $x_{i}=x_{j}=1$. ∎

5.4 Dominance relation among the new valid inequalities

We start this subsection investigating whether SCILS  dominates CILS  or vice versa.

Theorem 21**.**

Let $C$ be the cover in (29) and consider $\gamma:=|C|$ to be even.

If $\beta=\gamma-1$, then the sum of all inequalities in SCILS  is equivalent to CILS . Therefore, in this case, the set of inequalities in SCILS  dominates CILS . 2. 2.

If $\beta<\gamma-1$, there is no dominance relation between SCILS  and CILS .

Proof.

Let $sum(\hyperlink{scilsineq}{\textbf{SCILS}\,})$ denote the inequality obtained by adding all inequalities in SCILS , and let $rhs(sum(\hyperlink{scilsineq}{\textbf{SCILS}\,}))$ denote its right-hand side (rhs). We have that $rhs(sum(\hyperlink{scilsineq}{\textbf{SCILS}\,}))$ is equal to the number of inequalities in SCILS  multiplied by the rhs of each inequality, i.e.:

[TABLE]

The coefficient of each variable $X_{ij}$ in $sum(\hyperlink{scilsineq}{\textbf{SCILS}\,})$ ($coef_{ij}$) is given by the number of inequalities in the set SCILS  in which $X_{ij}$ appears, i.e.:

[TABLE]

Dividing $rhs(sum(\hyperlink{scilsineq}{\textbf{SCILS}\,}))$ by $coef_{ij}$, we obtain

[TABLE]

On the other side, the rhs of CILS  is:

[TABLE]

Replacing $\beta$ with $\gamma-1$, and $\left\lfloor\frac{\beta}{2}\right\rfloor$ with $\frac{\beta-1}{2}$ (since $\beta$ is odd), we obtain the result. 2. 2.

Consider, for example, $C=\{1,2,3,4,5,6\}$ and $\beta=3$ ($\beta<\gamma-1$ and odd). In this case, the CILS  becomes:

[TABLE]

And a particular inequality in SCILS  is

[TABLE]

The solution $X_{1j}=1$, for $j=2,\ldots,6$, and all other variables equal to zero, satisfies all inequalities in SCILS , because only one of the positive variables appears in each inequality in the set. However, the solution does not satisfy CILS . On the other side, the solution $X_{12}=X_{34}=X_{56}=1$, and all other variables equal to zero, satisfies CILS , but does not satisfy (40).

Now, consider $C=\{1,2,3,4,5,6\}$ and $\beta=4$ ($\beta<\gamma-1$ and even). In this case, the CILS  becomes:

[TABLE]

And a particular inequality in SCILS  is

[TABLE]

The solution $X_{1j}=1$, for $j=2,\ldots,6$, $X_{2j}=1$, for $j=3,\ldots,6$, and all other variables equal to zero, satisfies all inequalities in SCILS , because at most two of the positive variables appear in each inequality in the set. However, the solution does not satisfy CILS . On the other side, the solution $X_{12}=X_{34}=X_{56}=1$, and all other variables equal to zero, satisfies CILS , but does not satisfy (41).

∎

Theorem 22**.**

Let $C$ be the cover in (29) and consider $\gamma:=|C|$ to be odd. Then there is no dominance relation between SCILS  and CILS .

Proof.

Consider, for example, $C=\{1,2,3,4,5\}$ and $\beta=3$ ($\beta$ odd). In this case, the CILS  becomes:

[TABLE]

And a particular inequality in SCILS  is

[TABLE]

The solution $X_{1j}=1$, for $j=1,\ldots,5$, and all other variables equal to zero, satisfies all inequalities in SCILS , because only one of the positive variables appears in each inequality in the set. However, the solution does not satisfy CILS . On the other side, the solution $X_{23}=X_{45}=1$, and all other variables equal to zero, satisfies CILS , but does not satisfy (42).

Now, consider $C=\{1,2,3,4,5\}$ and $\beta=4$ ($\beta$ even). In this case, the CILS  becomes:

[TABLE]

And a particular inequality in SCILS  is

[TABLE]

The solution $X_{1j}=1$, for $j=1,\ldots,5$, $X_{2j}=1$, for $j=2,\ldots,5$, and all other variables equal to zero, satisfies all inequalities in SCILS , because at most two of the positive variables appear in each inequality in the set. However, the solution does not satisfy CILS . On the other side, the solution $X_{11}=X_{23}=X_{45}=1$, and all other variables equal to zero, satisfies CILS , but does not satisfy (43). ∎

Now, we investigate if SCILS  is just a particular case of SKILS , when $\alpha_{j}\in\{0,1\}$, for all $j\in N$ in (35).

Theorem 23**.**

In case the modification suggested in Remark 19 is applied, then if $|C|$ is even in (29), SCILS  becomes just a particular case of SKILS . In case $|C|$ is odd, however, the inequalities in SCILS  are stronger.

Proof.

If $|C|$ is even, the result is easily verified. If $|C|$ is odd, the inequalities in SCILS  become

[TABLE]

if $\beta$ is odd, and

[TABLE]

if $\beta$ is even, and the inequalities in SKILS  become

[TABLE]

for all $\beta$. In all cases, $C_{s}$ is a partition of $C\setminus\{i_{0}\}$, where $i_{0}\in C$.

Either with $\beta$ even or odd, it becomes clear that SCILS  is stronger than SKILS . ∎

6 Lower bounds from solutions of the relaxations for QKP ${}_{\mbox{lifted}}$

In order to evaluate the quality of the upper bounds obtained with CRel, we compare them with lower bounds for QKP, given by feasible solutions constructed by a heuristic. We assume in this section that all variables in CRel are constrained to the interval $[0,1]$.

Let $(\bar{x},\bar{X})$ be a solution of CRel. We initially apply principal component analysis (PCA) [19] to construct an approximation to the solution of QKP and then apply a special rounding procedure to obtain a feasible solution from it. PCA selects the largest eigenvalue and the corresponding eigenvector of $\bar{X}$, denoted by $\bar{\lambda}$ and $\bar{v}$, respectively. Then $\bar{\lambda}\bar{v}\bar{v}^{T}$ is a rank-one approximation of $\bar{X}$. We set $\bar{x}=\bar{\lambda}^{\frac{1}{2}}\bar{v}$ to be an approximation of the solution $x$ of QKP. We note that $\bar{\lambda}>0$ because $\bar{X}_{ii}>0$ for at least one index $i$ in the optimal solutions of the relaxations, and therefore, $\bar{X}$ is not negative semidefinite. Finally, we round $\bar{x}$ to a binary solution that satisfies the knapsack capacity constraint, using the simple approach described in Algorithm 3.

7 Numerical Experiments

We summarize our algorithm CWICS (Convexification With Integrated Cut Strengthening) in Algorithm 4, where at each iteration we update the perturbation $Q_{p}$ of the parametric relaxation and, at every $m$ iterations, we add to the relaxation, the valid inequalities considered in this paper, namely, SCI, defined in (28), CILS, defined in (30), and SCILS, defined in (32).

The numerical experiments performed had the following main purposes,

verify the impact of the valid inequalities, SCI, CILS, and SCILS, when iteratively added to cut the current solution of a relaxation of QKP,

• 2.

verify the effectiveness of the IPM described in Section 3 in decreasing the upper bound while optimizing the perturbation $Q_{p}$,

• 3.

compute the upper and lower bounds obtained with the proposed algorithmic approach described in CWICS (Algorithm 4), and compare them, with the optimal solutions of the instances.

We coded CWICS in MATLAB, version R2016b, and ran the code on a notebook with an Intel Core i5-4200U CPU 2.30GHz, 6GB RAM, running under Windows 10. We used the primal-dual IPM implemented in Mosek, version 8, to solve the relaxation CQPQp, and, to solve the separation problems MIQP 1 and MILP 2, we use Gurobi, version 8.

The input data used in the first iteration of the IPM described in Algorithm 1 ($k=0$) are: $B_{0}=I$, $\mu^{0}=1$. We start with a matrix $Q^{0}_{p}$, such that $Q-Q^{0}_{p}$ is negative definite. By solving CQPQp, with $Q_{p}:=Q^{0}_{p}$, we obtain $x(Q^{0}_{p})$, $X(Q^{0}_{p})$, as its optimal solution, and set $\nabla p^{*}_{\textsc{\tiny CQP}}(Q^{0}_{p}):=X(Q^{0}_{p})-x(Q^{0}_{p})x(Q^{0}_{p})^{T}$. Finally, the positive definiteness of $Z^{0}$ and $\Lambda^{0}$ are assured by setting: $Z^{0}:=Q^{0}_{p}-Q$ and $\Lambda^{0}:=\nabla p^{*}_{\textsc{\tiny CQP}}(Q^{0}_{p})+(2|\lambda_{\min}(\nabla p^{*}_{\textsc{\tiny CQP}}(Q^{0}_{p})|+0.1)I$.

Our randomly generated test instances were also used by J. O. Cunha in [8], who provided us with the instances data and with their optimal solutions. Each weight $w_{j}$, for $j\in N$, was randomly selected in the interval $[1,50]$, and the capacity $c$, of the knapsack, was randomly selected in $[50,\sum_{j=1}^{n}w_{j}]$. The procedure used by Cunha to generate the instances was based on previous works [5, 6, 7, 12, 22].

The following labels identify the results presented in Tables 3 and 5–6.

OptGap (%):= ((upper bound - opt)/opt) $\times$ 100, where opt is the optimal solution value (the relative optimality gap),

• 2.

Time (sec) (the computational time to compute the bound),

• 3.

DuGap (%) := (upper bound - lower bound)/(lower bound) $\times$ 100, where the lower bound is computed as described in Sect. 6 (the relative duality gap),

• 4.

Iter (the number of iterations),

• 5.

Cuts (the number of cuts added to the relaxation),

• 6.

Time${}_{\tiny{\mbox{MIP}}}$ (sec) (the computational time to obtain cuts CILS  and SCILS ).

To get some insight into the effectiveness of the cuts proposed, we initially applied them to 10 small instances with $n=10$. In Table 3 we present average results for this preliminary experiment, where we iteratively add the cuts to the following linear relaxation

[TABLE]

In the first row of Table 3, the results correspond to the solution of the linear relaxation $\tilde{\hyperlink{modellinsdp}{\textbf{LPR}\,}}$ with no cuts. In SCI1, we add only the most violated cut from the $n$ cuts in SCI  to $\tilde{\hyperlink{modellinsdp}{\textbf{LPR}\,}}$ at each iteration, and in the SCI  we add all $n$ cuts. In CILS  and SCILS, we solve MIQP and MILP problems to find the most violated cut of each type. The last row of the table (All) corresponds to results obtained when we add all $n$ cuts in SCI, and one cut of each type, CILS   and SCILS. In these initial tests, we run up to 50 iterations, and in most cases, stop the algorithm when no more cuts are found to be added to the relaxation.

Figure 1 depicts the optimality gaps from Table 3. There is a trade-off between the quality of the cuts and the computational time needed to find them. Considering a unique cut of each type, we note that SCILS  is the strongest cut (OptGap $=9.121\%$), but the computational time to obtain it, if compared to CILS  and SCI, is bigger. Nevertheless, a decrease in the times could be achieved with a heuristic solution for the separation problems, and also by the application of better stopping criteria for the cutting plane algorithm. We point out that using all cuts together we find a better upper bound than using each type of cut separately (OptGap $=3.315\%$).

We now analyze the effectiveness of our IPM in decreasing the upper bound while optimizing the perturbation $Q_{p}$. To improve the bounds obtained, besides the constraints in (44), we also consider in the initial relaxation, the valid inequalities $X_{ii}=x_{i}$, the McCormick inequalities $X_{ij}\leq X_{ii}$, and the valid inequalities obtained by multiplying the capacity constraint by each nonnegative variable $x_{i}$, and also by $(1-x_{i})$, and then replacing each bilinear term $x_{i}x_{j}$ by $X_{ij}$. We then start the algorithm solving the following relaxation.

[TABLE]

In order to evaluate the influence of the initial decomposition of $Q$ on the behavior of the IPM, we considered two initial decompositions. In both cases, we compute the eigendecomposition of $Q$, getting $Q=\sum_{i=1}^{n}\lambda_{i}v_{i}v_{i}^{\prime}$.

For the first decomposition, we set $Q_{n}:=\sum_{i=1}^{n}(-|\lambda_{i}|-1)v_{i}v_{i}^{\prime}$, and $Q^{0}_{p}:=Q-Q_{n}/2$. We refer to this initial matrix $Q_{p}^{0}$ as $Q_{p}^{a}$.

• 2.

For the second, we set $Q_{n}:=\sum_{i=1}^{n}(\min\{\lambda_{i},-10^{-6}\})v_{i}v_{i}^{\prime}$, and $Q^{0}_{p}:=Q-Q_{n}/2$. We refer to this initial matrix $Q_{p}^{0}$ as $Q_{p}^{b}$.

In Table 4, we compare the bounds obtained by our IPM after 20 iterations ($boundIPM_{20}$), with the bounds given from the linear SDP relaxation obtained by taking $Q_{p}^{0}=Q$ in (45), and adding to it the semidefinite constraint $X-xx^{\prime}\succeq 0$ ($boundSDP$). As mentioned in Section 4, these are the best possible bounds that can be obtained by the IPM algorithm. We also show in Table 4 how close to $boundSDP$, the bound computed with the initial decomposition $Q_{p}^{0}$ ($boundIPM_{1}$) in relaxation (45) is. The values presented in the table are

[TABLE]

For the experiment reported in Table 4, we consider 10 instances with $n=50$ and $100$. We see from the results in Table 4 that in 20 iterations, the IPM closed the gap to the SDP bounds for all instances. When starting from $Q_{p}^{a}$, we end up with an average bound less than 0.1% of the SDP bound, while when starting from $Q_{p}^{b}$, this percentage decreases to only 0.03%. We also start from better bounds when considering $Q_{p}^{b}$, and therefore, we use this matrix as the initial decomposition for the IPM in the next experiments. The results in Table 4 show that the IPM developed in this paper is effective to solve the parametric problem (6), converging to bounds very close to the solution of the SDP relaxation, which are their minimum possible values.

We finally present results obtained from the application of CWICS, considering the parametric quadratic relaxation, the IPM, and the cuts. In Tables 5 and 6 we show the results for the same instances with $n=50$ and $n=100$ considered in the previous experiment. The cuts are added at every $m$ iterations of the IPM and the numbers of cuts added at each iteration are $n$ SCI, 5 CILS  and 5 SCILS . Note that when solving each MIQP or MILP problem, besides the cut CILS  or SCILS , we also obtain a cover inequality CI . We check if this CI  was already added to the relaxation, and if not, we add it as well. We stop CWICS when a maximum number of iterations is reached or when DuGap is sufficiently small.

For the results presented in Table 5 ($n=50$), we set the maximum number of iterations of the IPM equal to 100, and $m=10$. The execution of each separation problem was limited to 3 seconds, and the best solutions obtained in this time limit was used to generate the cuts.

For the results presented in Table 6 ($n=100$), we set the maximum number of iterations of the IPM equal to 20, and $m=4$. In this case, the execution of each separation problem was limited to 10 seconds.

We note from the results in Tables 5 and 6, that the alternation between the iterations of the IPM to improve the perturbation $Q_{p}$ of the relaxation and the addition of cuts to the relaxation, changing the search direction of the IPM, is an effective approach to compute bounds for QKP. Considering the stopping criterion imposed to CWICS, it was able to converge to the optimum solution of three out of five instances with $n=50$ and of two out of five instances with $n=100$. The average optimality gap for all ten instances is less than $0.1\%$. The heuristic applied also computed good solutions for the problem. The average duality gap for the 10 instances is less than $0.25\%$.

We note that our algorithm spends a high percentage of its running time solving the separation problems, and also solving the linear systems to define the direction of improvement in the IPM algorithm. The running time of both procedures can be improved by a more judicious implementation. There are two parameters in CWICS that can also be better analyzed and tuned to improve the results, namely, $m$ and the time limit for the execution of the separation problems. As mentioned before, these problems could still be solved by heuristics. Finally, we note that the alternation between the IPM iterations and addition of cuts to the relaxation could be combined with a branch-and-bound algorithm in an attempt to converge faster to the optimal solution. In this case, the cuts added to the relaxations would include the cuts that define the branching and the update on $Q_{p}$ would depend on the branch of the enumeration tree. These are directions for the continuity of the research on this work.

8 Conclusion

In this paper we present an algorithm named CWICS (Convexification With Integrated Cut Strengthening), that iteratively improves the upper bound for the quadratic knapsack problem (QKP). The initial relaxation for the problem is given by a parametric convex quadratic problem, where the Hessian $Q$ of the objective function of the QKP is perturbed by a matrix parameter $Q_{p}$, such that $Q-Q_{p}\preceq 0$. Seeking for the best possible bound, the concave term $x^{T}(Q-Q_{p})x$, is then kept in the objective function of the relaxation and the remaining part, given by $x^{T}Q_{p}x$ is linearized through the standard approach that lifts the problem to space of symmetric matrices defined by $X:=xx^{T}$.

We present a primal-dual interior point method (IPM), which update the perturbation $Q_{p}$ at each iteration of CWICS aiming at reducing the upper bound given by the relaxation. We also present new classes of cuts that are added during the execution of CWICS, which are defined on the lifted variable $X$, and derived from cover inequalities and the binary constraints.

We show that both the IPM and the cuts generated are effective in improving the upper bound for the QKP and apply them in an integrated procedure that alternates between the optimization of the perturbation $Q_{p}$ and the strengthening of the feasible set of the relaxation with the addition of the cuts. We note that these procedures could be applied to more general binary indefinite quadratic problems as well. The separation problems described to generate the cuts could also be solved heuristically, in order to accelerate the process.

We note that the search for the best perturbation $Q_{p}$, by our IPM, is updated with the inclusion of cuts to the relaxation. In the set of cuts added, we could also consider cuts defined by the branching procedure in a branch-and-bound algorithm. In this case, we could have the perturbation $Q_{p}$ optimized during all the descend on the branch-and-bound tree, considering the cuts that had been added to the relaxations.

Finally, we show that if the positive semidefinite constraint $X-xx^{T}\succeq 0$ was introduced in the relaxation of the QKP, or any other indefinite quadratic problem (maximizing the objective function), then the decomposition of objective function, that leads to a convex quadratic SDP relaxation, where a perturbed concave part of the objective is kept, and the remaining part is linearized, is not effective. In this case the best bound is always attained when the whole objective function is linearized, i.e., when the perturbation $Q_{p}$ is equal to $Q$. This observation also relates to the well known DC (difference of convex) decomposition of indefinite quadratics that have been used in the literature to generate bounds for indefinite quadratic problems. Once more, in case the positive semidefinite constraint is added to the relaxation, the DC decomposition is not effective anymore, and the alternative linear SDP relaxation leads to the best possible bound. As corollary from this result, we see that the bound given by the convex quadratic relaxation cannot be better than the bound given by the corresponding linear SDP relaxation.

Acknowledgements

We thank J. O. Cunha for providing us with the test instances used in our experiments. M. Fampa was partially supported by CNPq (National Council for Scientific and Technological Development) Grant 303898/2016-0, D. C. Lubke was supported by Research Grants from CAPES (Brazilian Federal Agency for Support and Evaluation of Graduate Education) Grant 88881.131629/2016-01, CNPq Grant 142143/2015-4 and partially supported by PSR - Power Systems Research, Rio de Janeiro, Brazil.