Learning to Branch

TL;DR

This paper introduces a machine learning approach to optimize tree search algorithms by learning the best combination of partitioning procedures, significantly reducing tree size and improving efficiency with theoretical guarantees.

Contribution

It presents the first theoretical sample complexity guarantees for tree search algorithm configuration and demonstrates a practical learning method to optimize branching.

Findings

Learning optimal partitioning weights reduces tree size dramatically.

The reduction in tree size can be exponential.

The approach is both theoretically sound and practically effective.

Abstract

Tree search algorithms, such as branch-and-bound, are the most widely used tools for solving combinatorial and nonconvex problems. For example, they are the foremost method for solving (mixed) integer programs and constraint satisfaction problems. Tree search algorithms recursively partition the search space to find an optimal solution. In order to keep the tree size small, it is crucial to carefully decide, when expanding a tree node, which question (typically variable) to branch on at that node in order to partition the remaining space. Numerous partitioning techniques (e.g., variable selection) have been proposed, but there is no theory describing which technique is optimal. We show how to use machine learning to determine an optimal weighting of any set of partitioning procedures for the instance distribution at hand using samples from the distribution. We provide the first sample complexity guarantees for tree search algorithm configuration. These guarantees bound the number of samples sufficient to ensure that the empirical performance of an algorithm over the samples nearly matches its expected performance on the unknown instance distribution. This thorough theoretical investigation naturally gives rise to our learning algorithm. Via experiments, we show that learning an optimal weighting of partitioning procedures can dramatically reduce tree size, and we prove that this reduction can even be exponential. Through theory and experiments, we show that learning to branch is both practical and hugely beneficial.

Figures

Equations

$$\begin{array}[]{lll}\textnormal{maximize}&\sum_{i=1}^{n}\sum_{b\in B_{i}}v_{i}(b)x_{i,b}&\\ \textnormal{s.t.}&\sum_{i=1}^{n}\sum_{b\in B_{i},j\ni b}x_{i,b}\leq 1&\forall j\in[m]\\ &\sum_{b\in B_{i}}x_{i,b}\leq 1&\forall i\in[n]\\ &x_{i,b}\in\{0,1\}&\forall i\in[n],b\in B_{i}.\end{array}$$

$$\begin{array}[]{ll}\text{maximize}&40x_{1}+60x_{2}+10x_{3}+10x_{4}+3x_{5}+20x_{6}+60x_{7}\\ \text{subject to}&40x_{1}+50x_{2}+30x_{3}+10x_{4}+10x_{5}+40x_{6}+30x_{7}\leq 100\\ &x_{1},\dots,x_{7}\in\{0,1\}.\end{array}$$

$$e(x)=\{\begin{array}{ll}-x{\log }_2(x)-(1-x){\log }_2(1-x)& \text{if }x\in (0,1)\\ 0& \text{if }x\in \{0,1\}.\end{array}$$

$${\text{score}}_1(\mathcal{T},Q,i)=\min \left\{{\breve{c}}_Q-{\breve{c}}_{Q_i^{+}},{\breve{c}}_Q-{\breve{c}}_{Q_i^{-}}\right\}\text{, }{\text{score}}_2(\mathcal{T},Q,i)=\max \left\{{\breve{c}}_Q-{\breve{c}}_{Q_i^{+}},{\breve{c}}_Q-{\breve{c}}_{Q_i^{-}}\right\},$$

$${\mathbb{E}}_{Q\sim \mathcal{D}}\left[\text{cost}\left(Q,\mu {\text{score}}_1+(1-\mu ){\text{score}}_2\right)\right]=\Omega \left(2^{(n-9)/4}\right)$$

$${\text{score}}_1(\mathcal{T},Q,i)=\min \left\{{\breve{c}}_Q-{\breve{c}}_{Q_i^{+}},{\breve{c}}_Q-{\breve{c}}_{Q_i^{-}}\right\}\text{, }{\text{score}}_2(\mathcal{T},Q,i)=\max \left\{{\breve{c}}_Q-{\breve{c}}_{Q_i^{+}},{\breve{c}}_Q-{\breve{c}}_{Q_i^{-}}\right\},$$

$$\begin{array}[]{ll}\text{maximize}&\bm{c}\cdot\bm{x}\\ \text{subject to}&2\sum_{i=1}^{n^{\prime}}x_{i}=n^{\prime}\\ &\bm{x}\in\{0,1\}^{n^{\prime}}.\end{array}$$

$$\begin{array}[]{ll}\text{maximize}&\bm{c}\cdot\bm{x}\\ \text{subject to}&2\sum_{i=1}^{n-3}x_{i}=n-3\\ &2\left(x_{n-2}+x_{n-1}+x_{n}\right)=3\\ &\bm{x}\in\{0,1\}^{n}.\end{array}$$

$$\begin{array}[]{ll}\text{maximize}&\left(1,2,3,4,5,0,\frac{3}{2},3-\frac{1}{2\mu^{*}}\right)\cdot\bm{x}\\ \text{subject to}&\begin{pmatrix}2&2&2&2&2&0&0&0\\ 0&0&0&0&0&2&2&2\end{pmatrix}\bm{x}=\begin{pmatrix}5\\ 3\end{pmatrix}\\ &\bm{x}\in\{0,1\}^{8}.\end{array}$$

$$\text{score}(\mathcal{T},Q,i)=\text{score}({\mathcal{T}}_Q,Q,i)$$

$$i={\text{argmax}}_j\left\{{\mu }^{\prime }{\text{score}}_1({\mathcal{T}}_Q,Q,j)+(1-{\mu }^{\prime }){\text{score}}_2({\mathcal{T}}_Q,Q,j)\right\},$$

$$i={\text{argmax}}_j\left\{{\mu }^{\prime \prime }{\text{score}}_1({\mathcal{T}}_Q,Q,j)+(1-{\mu }^{\prime \prime }){\text{score}}_2({\mathcal{T}}_Q,Q,j)\right\}.$$

$$i={\text{argmax}}_j\left\{{\mu }^{\prime }{\text{score}}_1(\mathcal{T},Q,j)+(1-{\mu }^{\prime }){\text{score}}_2(\mathcal{T},Q,j)\right\}$$

$$\left|{\mathbb{E}}_{z\sim \mathcal{D}}[f(z)]-\frac{1}{m}\sum _{i=1}^{m}f\left(z_i\right)\right|=O\left(\kappa \sqrt{\frac{\text{Pdim}\left(\mathcal{F}\right)}{m}}+\kappa \sqrt{\frac{\ln \left(1/\delta \right)}{m}}\right).$$

$$\left|{\mathbb{E}}_{Q\sim \mathcal{D}}[\text{cost}(Q,\mu {\text{score}}_1+\cdots +{\mu }_d{\text{score}}_d)]-\frac{1}{m}\sum _{i=1}^m\text{cost}(Q_i,\mu {\text{score}}_1+\cdots +{\mu }_d{\text{score}}_d)\right|$$

$$=$$

$${\hat{\mathcal{R}}}_{\mathcal{S}}(\mathcal{F})=\frac{1}{m}{\mathbb{E}}_{\mathbf{\sigma }\sim \{-1,1{\}}^m}\left[\underset{f\in \mathcal{F}}{\sup }\sum _{i=1}^m\sigma [i]f\left(z_i\right)\right].$$

$${\hat{\mathcal{R}}}_{\mathcal{S}}(\mathcal{F})\le \underset{\lambda >0}{\inf }\frac{1}{\lambda }\log \left(\sum _{\mathbf{a}\in \mathcal{F}(\mathcal{S})}\exp \left(\frac{1}{2}{\left(\frac{\lambda ||\mathbf{a}|{|}_2}{m}\right)}^2\right)\right).$$

$$C=\left\{((x_1,x_2),\psi ),((x_1,x_3),\psi ),((x_2,x_3),\psi ),((x_3,x_4),\psi )\right).$$

$$\text{cost}\left(Q,\mu {\text{score}}_1+(1-\mu ){\text{score}}_2\right)=O(1).$$

$$\begin{array}[]{ll}\text{maximize}&\bm{c}\cdot\bm{x}\\ \text{subject to}&A\bm{x}=(n-3,3)^{\top}\\ &\bm{x}\in\{0,1\}^{n}.\end{array}$$

$$\begin{array}[]{ll}\text{maximize}&\left(1,2,3,4,5,0,1.5,3-\frac{1}{2\mu^{*}}\right)\cdot\bm{x}\\ \text{subject to}&\begin{pmatrix}2&2&2&2&2&0&0&0\\ 0&0&0&0&0&2&2&2\end{pmatrix}\bm{x}=\begin{pmatrix}5\\ 3\end{pmatrix}\\ &\bm{x}\in\{0,1\}^{8}.\end{array}$$

$${\breve{x}}_Q[j]=\{\begin{array}{ll}0& \text{if }j\le \lfloor (n-3)/2\rfloor \text{ or }j=n-2\\ \frac{1}{2}& \text{if }j=\lceil (n-3)/2\rceil \text{ or }j=n-1\\ 1& \text{if }\lceil (n-3)/2\rceil \le j\le n-3\text{ or }j=n.\end{array}$$

$${\breve{c}}_{Q_i^{-}}$$

$${\breve{\mathbf{c}}}_{Q_i^{+}}$$

$${\breve{c}}_{Q_{n-1}^{-}}$$

$${\breve{c}}_{Q_{n-1}^{+}}$$

$$\mu {\text{score}}_1\left(Q_{n-1}^{-},n-2\right)+(1-\mu ){\text{score}}_2\left(Q_{n-1}^{-},n-2\right)$$

$$=B+\mu \left(\frac{3\gamma }{2}-\frac{\gamma }{4{\mu }^{\ast }}-B\right)$$

$$\ge B+{\mu }^{\ast }\left(\frac{3\gamma }{2}-\frac{\gamma }{4{\mu }^{\ast }}-B\right)$$

$$=B-\frac{\gamma }{4}+{\mu }^{\ast }\left(\frac{3\gamma }{2}-B\right)$$

$$>B-\frac{\gamma }{4}+\frac{3\gamma /4-B}{3\gamma /2-B}\left(\frac{3\gamma }{2}-B\right)$$

$$=B-\frac{\gamma }{4}+\frac{3\gamma }{4}-B$$

$$=\frac{\gamma }{2},$$

$$\mu {\text{score}}_1\left(Q^{\prime },p_{i^{\prime }+1}\right)+(1-\mu ){\text{score}}_2\left(Q^{\prime },p_{i^{\prime }+1}\right)$$

Full text

1 Introduction

Many widely-used algorithms are customizable: they have tunable parameters that have an enormous effect on runtime, solution quality, or both. Tuning parameters by hand is notoriously tedious and time-consuming. In this work, we study algorithm configuration via machine learning, where the goal is to design algorithms that learn the optimal parameter setting for the problem instance distribution at hand.

We study configuration of tree search algorithms. These algorithms are the most widely used tools for solving combinatorial and nonconvex problems throughout artificial intelligence, operations research, and beyond (e.g., (Russell and Norvig, 2010; Williams, 2013)). For example, branch-and-bound (B&B) algorithms (Land and Doig, 1960) solve mixed integer linear programs (MILPs), and thus have diverse applications, including ones in machine learning such as MAP estimation (Kappes et al., 2013), object recognition (Kokkinos, 2011), clustering (Komodakis et al., 2009), and semi-supervised SVMs (Chapelle et al., 2007).

A tree search algorithm systematically partitions the search space to find an optimal solution. The algorithm organizes this partition via a tree: the original problem is at the root and the children of a given node represent the subproblems formed by partitioning the feasible set of the parent node. A branch is pruned if it is infeasible or it cannot produce a better solution than the best one found so far by the algorithm. Typically the search space is partitioned by adding an additional constraint on some variable. For example, suppose the feasible set is defined by the constraint $A\bm{x}\leq\bm{b}$, with $\bm{x}\in\{0,1\}^{n}$. A tree search algorithm might partition this feasible set into two sets, one where $A\bm{x}\leq\bm{b}$, $x_{1}=0$, and $x_{2},\dots,x_{n}\in\{0,1\}$, and another where $A\bm{x}\leq\bm{b}$, $x_{1}=1$, and $x_{2},\dots,x_{n}\in\{0,1\}$, in which case the algorithm has branched on $x_{1}$. A crucial question in tree search algorithm design is determining which variable to branch on at each step. An effective variable selection policy can have a tremendous effect on the size of the tree. Currently, there is no known optimal strategy and the vast majority of existing techniques are backed only by empirical comparisons. In the worst-case, finding an approximately optimal branching variable, even at the root of the tree alone, is NP-hard. This is true even in the case of satisfiability, which is a special case of constraint satisfaction and of MILP (Liberatore, 2000).

In this work, rather than attempt to characterize a branching strategy that is universally optimal, we show empirically and theoretically that it is possible to learn high-performing branching strategies for a given application domain. We model an application domain as a distribution over problem instances, such as a distribution over scheduling problems that an airline solves on a day-to-day basis. This model is standard throughout the algorithm configuration literature (e.g., (Hutter et al., 2009, 2011; Dai et al., 2017; Kleinberg et al., 2017)). The approach has also been used on large-scale problems in industry to configure and select winner determination algorithms for clearing tens of billions of dollars of combinatorial auctions (Sandholm, 2013). The algorithm designer does not know the underlying distribution over problem instances, but has sample access to the distribution. We show how to use samples from the distribution to learn a variable selection policy that will result in as small a search tree as possible in expectation over the underlying distribution.

Our learning algorithm adaptively partitions the parameter space of the variable selection policy into regions where for any parameter in a given region, the resulting tree sizes across the training set are invariant. The learning algorithm returns the empirically optimal parameter over the training set, and thus performs empirical risk minimization (ERM). We prove that the adaptive nature of our algorithm is necessary: performing ERM over a data-independent discretization of the parameter space can be disastrous. In particular, for any discretization of the parameter space, we provide an infinite family of distributions over MILP instances such that every point in the discretization results in a B&B tree with exponential size in expectation, but there exist infinitely-many parameters outside of the discretized points that result in a tree with constant size with probability 1. A small change in parameters can thus cause a drastic change in the algorithm’s behavior. This fact contradicts conventional wisdom. For example, SCIP, the best open-source MILP solver, sets one of the parameters we investigate to 5/6, regardless of the input MILP’s structure. Achterberg (2009) wrote that 5/6 was empirically optimal when compared against four other data-independent values. In contrast, our analysis shows that a data-driven approach to parameter tuning can have an enormous benefit.

The sensitivity of tree search algorithms to small changes in their parameters is a key challenge that differentiates our sample complexity analysis from those typically found in machine learning. For many well-understood function classes in machine learning, there is a close connection between the distance in parameter space between two parameter vectors and the distance in function space between the two corresponding functions. Understanding this connection is a necessary prerequisite to analyzing how many significantly different functions there are in the class, and thereby quantifying the class’s intrinsic complexity. Intrinsic complexity typically translates to VC dimension, Rademacher complexity, or some other metric which allows us to derive learnability guarantees. Since the tree size of a search algorithm as a function of its parameters does not exhibit this predictable behavior, we must carefully analyze the way in which the parameters influence each step of the procedure in order to derive learning algorithms with strong guarantees. In doing so, we present the first sample complexity guarantees for automated configuration of tree search algorithms. We provide worst-case bounds proving that a surprisingly small number of samples are sufficient for strong learnability guarantees: the sample complexity bound grows quadratically in the size of the problem instance, despite the complexity of the algorithms we study.

In our experiments section, we show that on many datasets based on real-world NP-hard problems, different parameters can result in B&B trees of vastly different sizes. Using an optimal parameter for one distribution on problems from a different distribution can lead to a dramatic tree size blowup. We also provide data-dependent generalization guarantees that allow the algorithm designer to use far fewer samples than in the worst case if the data is well-structured.

1.1 Related work

Several works have studied the use of machine learning techniques in the context of B&B; for an overview, see the summary by Lodi and Zarpellon (2017).

As in this work, Khalil et al. (2016) study variable selection policies. Their goal is to find a variable selection strategy that mimics the behavior of the classic branching strategy known as strong branching while running faster than strong branching. Alvarez et al. (2017) study a similar problem, although in their work, the feature vectors in the training set describe nodes from multiple MILP instances. Neither or these works come with any theoretical guarantees, unlike our work.

Several other works study data-driven variable selection from a purely experimental perspective. Di Liberto et al. (2016) devise an algorithm that learns how to dynamically switch between different branching heuristics along the branching tree. Karzan et al. (2009) propose techniques for choosing problem-specific branching rules based on a partial B&B tree. Ideally, these branching rules will choose variables that will lead to fast fathoming. They do not rely on any techniques from machine learning. In the context of CSP tree search, Xia and Yap (2018) apply existing multi-armed bandit algorithms to learning variable selection policies during tree search and Balafrej et al. (2015) use a bandit approach to select different levels of propagation during search.

Other works have explored the use of machine learning techniques in the context of other aspects of B&B beyond variable selection. For example, He et al. (2014) use machine learning to speed up branch-and-bound, focusing on speeding up the node selection policy. Their work does not provide any learning-theoretic guarantees. Other works that have studied machine learning techniques for branch-and-bound problems other than variable selection include Sabharwal et al. (2017), who also study how to devise node selection policies, Hutter et al. (2009), who study how to set CPLEX parameters, Kruber et al. (2017), who study how to detect decomposable model structure, and Khalil et al. (2017), who study how to determine when to run heuristics.

From a theoretical perspective, Le Bodic and Nemhauser (2017) present a theoretical model for the selection of branching variables. It is based upon an abstraction of MIPs to a simpler setting in which it is possible to analytically evaluate the dual bound improvement of choosing a given variable. Based on this model, they present a new variable selection policy which has strong performance on many MIPLIB instances. Unlike our work, this paper is unrelated to machine learning.

The learning-theoretic model of algorithm configuration that we study in this paper was introduced to the theoretical computer science community by Gupta and Roughgarden (2017). Under this model, an application domain is modeled as a distribution over problem instances and the goal is to PAC-learn an algorithm that is nearly optimal over the distribution. This model was later studied by Balcan et al. (2017) as well. These papers were purely theoretical. In contrast, we show that the techniques proposed in this paper are practical as well, and provide significant benefit.

We provide a more detailed description of several of these papers in Appendix A.

2 Tree search

Tree search is a broad family of algorithms with diverse applications. To exemplify the specifics of tree search, we present a vast family of NP-hard problems — (mixed) integer linear programs — and describe how tree search finds optimal solutions to problems from this family. Later on in Section 5, we provide another example of tree search for constraint satisfaction problems. In Appendix E, we provide a formal, more abstract definition of tree search and generalize our results to this more general algorithm.

2.1 Mixed integer linear programs

We study mixed integer linear programs (MILPs) where the objective is to maximize $\bm{c}^{\top}\bm{x}$ subject to $A\bm{x}\leq\bm{b}$ and where some of the entries of $\bm{x}$ are constrained to be in $\{0,1\}$. Given a MILP $Q$, we denote an optimal solution to the LP relaxation of $Q$ as $\breve{\bm{x}}_{Q}=\left(\breve{x}_{Q}[1],\dots\breve{x}_{Q}[n]\right)$. Throughout this work, given a vector $\bm{a}$, we use the notation $a[i]$ to denote the $i^{th}$ component of $\bm{a}$. We also use the notation $\breve{c}_{Q}$ to denote the optimal objective value of the LP relaxation of $Q$. In other words, $\breve{c}_{Q}=\bm{c}^{\top}\breve{\bm{x}}_{Q}$.

Example 2.1** (Winner determination).**

Suppose there is a set $\{1,\dots,m\}$ of items for sale and a set $\{1,\dots,n\}$ of buyers. In a combinatorial auction, each buyer $i$ submits bids $v_{i}(b)$ for any number of bundles $b\subseteq\{1,\dots,m\}$. The goal of the winner determination problem is to allocate the goods among the bidders so as to maximize social welfare, which is the sum of the buyers’ values for the bundles they are allocated. We can model this problem as a MILP by assigning a binary variable $x_{i,b}$ for every buyer $i$ and every bundle $b$ they submit a bid $v_{i}(b)$ on. The variable $x_{i,b}$ is equal to 1 if and only if buyer $i$ receives the bundle $b$. Let $B_{i}$ be the set of all bundles $b$ that buyer $i$ submits a bid on. An allocation is feasible if it allocates no item more than once ($\sum_{i=1}^{n}\sum_{b\in B_{i},j\ni b}x_{i,b}\leq 1$ for all $j\in\{1,\dots,m\}$) and if each bidder receives at most one bundle ($\sum_{b\in B_{i}}x_{i,b}\leq 1$ for all $i\in\{1,\dots,n\}$). Therefore, the MILP is:

[TABLE]

2.1.1 MILP tree search

MILPs are typically solved using a tree search algorithm called branch-and-bound (B&B). Given a MILP problem instance, B&B relies on two subroutines that efficiently compute upper and lower bounds on the optimal value within a given region of the search space. The lower bound can be found by choosing any feasible point in the region. An upper bound can be found via a linear programming relaxation. The basic idea of B&B is to partition the search space into convex sets and find upper and lower bounds on the optimal solution within each. The algorithm uses these bounds to form global upper and lower bounds, and if these are equal, the algorithm terminates, since the feasible solution corresponding to the global lower bound must be optimal. If the global upper and lower bounds are not equal, the algorithm refines the partition and repeats.

In more detail, suppose we want to use B&B to solve a MILP $Q^{\prime}$. B&B iteratively builds a search tree $\mathcal{T}$ with the original MILP $Q^{\prime}$ at the root. In the first iteration, $\mathcal{T}$ consists of a single node containing the MILP $Q^{\prime}$. At each iteration, B&B uses a node selection policy (which we expand on later) to select a leaf node of the tree $\mathcal{T}$, which corresponds to a MILP $Q$. B&B then uses a variable selection policy (which we expand on in Section 2.1.2) to choose a variable $x_{i}$ of the MILP $Q$ to branch on. Specifically, let $Q^{+}_{i}$ (resp., $Q^{-}_{i}$) be the MILP $Q$ except with the additional constraint that $x_{i}=1$ (resp., $x_{i}=0$). B&B sets the right (resp., left) child of $Q$ in $\mathcal{T}$ to be a node containing the MILP $Q^{+}_{i}$ (resp., $Q^{-}_{i}$). B&B then tries to “fathom” these leafs: the leaf containing $Q^{+}_{i}$ (resp., $Q^{-}_{i}$) is fathomed if:

The optimal solution to the LP relaxation of $Q^{+}_{i}$ (resp., $Q^{-}_{i}$) satisfies the constraints of the original MILP $Q^{\prime}$. 2. 2.

The relaxation of $Q^{+}_{i}$ (resp., $Q^{-}_{i}$) is infeasible, so $Q^{+}_{i}$ (resp., $Q^{-}_{i}$) must be infeasible as well. 3. 3.

The objective value of the LP relaxation of $Q^{+}_{i}$ (resp., $Q^{-}_{i}$) is smaller than the objective value of the best known feasible solution, so the optimal solution to $Q^{+}_{i}$ (resp., $Q^{-}_{i}$) is no better than the best known feasible solution.

B&B terminates when every leaf has been fathomed. It returns the best known feasible solution, which is optimal. See Algorithm 1 for the pseudocode.

The most common node selection policy is the best bound policy. Given a B&B tree, it selects the unfathomed leaf containing the MILP $Q$ with the maximum LP relaxation objective value. Another common policy is the depth-first policy, which selects the next unfathomed leaf in the tree in depth-first order.

Example 2.2**.**

In Figure 1, we show the search tree built by B&B given as input the following MILP (Kolesar, 1967):

[TABLE]

Each rectangle denotes a node in the B&B tree. Given a node $Q$, the top portion of its rectangle displays the optimal solution $\breve{\bm{x}}_{Q}$ to the LP relaxation of $Q$, which is the MILP (1) with the additional constraints labeling the edges from the root to $Q$. The bottom portion of the rectangle corresponding to $Q$ displays the objective value $\breve{c}_{Q}$ of the optimal solution to this LP relaxation, i.e., $\breve{c}_{Q}=(40,60,10,10,3,20,60)\cdot\breve{\bm{x}}_{Q}$. In this example, the node selection policy is the best bound policy and the variable selection policy selects the “most fractional” variable: the variable $x_{i}$ such that $\breve{x}_{Q}[i]$ is closest to $\frac{1}{2}$, i.e., $i=\textnormal{argmax}\left\{\min\left\{1-\breve{x}_{Q}[i],\breve{x}_{Q}[i]\right\}\right\}$.

In Figure 1, the algorithm first explores the root. At this point, it has the option of exploring either the left or the right child. Since the optimal objective value of the right child (136) is greater than the optimal objective value of the left child (135), B&B will next explore the pink node (marked
1 ). Next, B&B can either explore either of the pink node’s children or the orange node (marked
2 ). Since the optimal objective value of the orange node (135) is greater than the optimal objective values of the pink node’s children (120), B&B will next explore the orange node. After that B&B can explore either of the orange node’s children or either of the pink node’s children. The optimal objective value of the green node (marked
3 ) is higher than the optimal objective values of the orange node’s right child (116) and the pink node’s children (120), so B&B will next explore the green node. At this point, it finds an integral solution, which satisfies all of the constraints of the original MILP (1). This integral solution has an objective value of 133. Since all of the other leafs have smaller objective values, the algorithm cannot find a better solution by exploring those leafs. Therefore, the algorithm fathoms all of the leafs and terminates.

2.1.2 Variable selection in MILP tree search

Variable selection policies typically depend on a real-valued score per variable $x_{i}$.

Definition 2.1** (Score-based variable selection policy).**

Let score be a deterministic function that takes as input a partial search tree $\mathcal{T}$, a leaf $Q$ of that tree, and an index $i$ and returns a real value ($\texttt{score}(\mathcal{T},Q,i)\in\mathbb{R}$). For a leaf $Q$ of a tree $\mathcal{T}$, let $N_{\mathcal{T},Q}$ be the set of variables that have not yet been branched on along the path from the root of $\mathcal{T}$ to $Q$. A score-based variable selection policy selects the variable $\textnormal{argmax}_{x_{j}\in N_{\mathcal{T},Q}}\{\texttt{score}(\mathcal{T},Q,j)\}$ to branch on at the node $Q$.

We list several common definitions of the function score below. Recall that for a MILP $Q$ with objective function $\bm{c}\cdot\bm{x}$, we denote an optimal solution to the LP relaxation of $Q$ as $\breve{\bm{x}}_{Q}=\left(\breve{x}_{Q}[1],\dots\breve{x}_{Q}[n]\right)$. We also use the notation $\breve{c}_{Q}$ to denote the objective value of the optimal solution to the LP relaxation of $Q$, i.e., $\breve{c}_{Q}=\bm{c}^{\top}\breve{\bm{x}}_{Q}$. Finally, we use the notation $Q_{i}^{+}$ (resp., $Q_{i}^{-}$) to denote the MILP $Q$ with the additional constraint that $x_{i}=1$ (resp., $x_{i}=0$). If $Q_{i}^{+}$ (resp., $Q_{i}^{-}$) is infeasible, then we set $\breve{c}_{Q}-\breve{c}_{Q_{i}^{+}}$ (resp., $\breve{c}_{Q}-\breve{c}_{Q_{i}^{-}}$) to be some large number greater than $||\bm{c}||_{1}$.

Most fractional.

In this case, $\texttt{score}(\mathcal{T},Q,i)=\min\left\{1-\breve{x}_{Q}[i],\breve{x}_{Q}[i]\right\}.$ The variable that maximizes $\texttt{score}(\mathcal{T},Q,i)$ is the “most fractional” variable, since it is the variable such that $\breve{x}_{Q}[i]$ is closest to $\frac{1}{2}$.

Linear scoring rule (Linderoth and Savelsbergh, 1999).

In this case, $\texttt{score}(\mathcal{T},Q,i)=(1-\mu)\cdot\max\left\{\breve{c}_{Q}-\breve{c}_{Q_{i}^{+}},\breve{c}_{Q}-\breve{c}_{Q_{i}^{-}}\right\}+\mu\cdot\min\left\{\breve{c}_{Q}-\breve{c}_{Q_{i}^{+}},\breve{c}_{Q}-\breve{c}_{Q_{i}^{-}}\right\}$ where $\mu\in[0,1]$ is a user-specified parameter. This parameter balances an “optimistic” and a “pessimistic” approach to branching: An optimistic approach would choose the variable that maximizes $\max\left\{\breve{c}_{Q}-\breve{c}_{Q_{i}^{+}},\breve{c}_{Q}-\breve{c}_{Q_{i}^{-}}\right\}$, which corresponds to $\mu=0$, and a pessimistic approach would choose the variable that maximizes $\min\left\{\breve{c}_{Q}-\breve{c}_{Q_{i}^{+}},\breve{c}_{Q}-\breve{c}_{Q_{i}^{-}}\right\}$, which corresponds to $\mu=1$.

Product scoring rule (Achterberg, 2009).

In this case, $\texttt{score}(\mathcal{T},Q,i)=\max\left\{\breve{c}_{Q}-\breve{c}_{Q_{i}^{-}},\gamma\right\}\cdot\max\left\{\breve{c}_{Q}-\breve{c}_{Q_{i}^{+}},\gamma\right\}$ where $\gamma=10^{-6}$. Comparing $\breve{c}_{Q}-\breve{c}_{Q_{i}^{-}}$ and $\breve{c}_{Q}-\breve{c}_{Q_{i}^{+}}$ to $\gamma$ allows the algorithm to compare two variables even if $\breve{c}_{Q}-\breve{c}_{Q_{i}^{-}}=0$ or $\breve{c}_{Q}-\breve{c}_{Q_{i}^{+}}=0$. After all, suppose the scoring rule simply calculated the product $\left(\breve{c}_{Q}-\breve{c}_{Q_{i}^{-}}\right)\cdot\left(\breve{c}_{Q}-\breve{c}_{Q_{i}^{+}}\right)$ without comparing to $\gamma$. If $\breve{c}_{Q}-\breve{c}_{Q_{i}^{-}}=0$, then the score equals 0, canceling out the value of $\breve{c}_{Q}-\breve{c}_{Q_{i}^{+}}$ and thus losing the information encoded by this difference.

Entropic lookahead scoring rule (Gilpin and Sandholm, 2011).

Let

[TABLE]

Set $\texttt{score}(\mathcal{T},Q,i)=-\sum_{j=1}^{n}\left(1-\breve{x}_{Q}[i]\right)\cdot e\left(\breve{x}_{Q_{i}^{-}}[j]\right)+\breve{x}_{Q}[i]\cdot e\left(\breve{x}_{Q_{i}^{+}}[j]\right).$

Alternative definitions of the linear and product scoring rules.

In practice, it is often too slow to compute the differences $\breve{c}_{Q}-\breve{c}_{Q_{i}^{-}}$ and $\breve{c}_{Q}-\breve{c}_{Q_{i}^{+}}$ for every variable, since it requires solving as many as $2n$ LPs. A faster option is to partially solve the LP relaxations of $Q_{i}^{-}$ and $Q_{i}^{+}$, starting at $\breve{\bm{x}}_{Q}$ and running a small number of simplex iterations. Denoting the new objective values as $\tilde{c}_{Q_{i}^{-}}$ and $\tilde{c}_{Q_{i}^{+}}$, we can revise the linear scoring rule to be $\texttt{score}(\mathcal{T},Q,i)=(1-\mu)\cdot\max\left\{\breve{c}_{Q}-\tilde{c}_{Q_{i}^{+}},\breve{c}_{Q}-\tilde{c}_{Q_{i}^{-}}\right\}+\mu\cdot\min\left\{\breve{c}_{Q}-\tilde{c}_{Q_{i}^{+}},\breve{c}_{Q}-\tilde{c}_{Q_{i}^{-}}\right\}$ and we can revise the product scoring rule to be $\texttt{score}(\mathcal{T},Q,i)=\max\left\{\breve{c}_{Q}-\tilde{c}_{Q_{i}^{-}},\gamma\right\}\cdot\max\left\{\breve{c}_{Q}-\tilde{c}_{Q_{i}^{+}},\gamma\right\}$. Other popular alternatives to computing $\breve{c}_{Q_{i}^{-}}$ and $\breve{c}_{Q_{i}^{+}}$ that fit within our framework are pseudo-cost branching (Bénichou et al., 1971; Gauthier and Ribière, 1977; Linderoth and Savelsbergh, 1999) and reliability branching (Achterberg et al., 2005).

3 Guarantees for data-driven learning to branch

In this section, we begin with our formal problem statement. We then present worst-case distributions over MILP instances demonstrating that learning over any data-independent discretization of the parameter space can be inadequate. Finally, we present sample complexity guarantees and a learning algorithm. Throughout the remainder of this paper, we assume that all aspects of the tree search algorithm except the variable selection policy, such as the node selection policy, are fixed.

3.1 Problem statement

Let $\mathcal{D}$ be a distribution over MILPs $Q$. For example, $\mathcal{D}$ could be a distribution over clustering problems a biology lab solves day to day, formulated as MILPs. Let $\texttt{score}_{1},\dots,\texttt{score}_{d}$ be a set of variable selection scoring rules, such as those in Section 2.1.2. Our goal is to learn a convex combination $\mu_{1}\texttt{score}_{1}+\cdots+\mu_{d}\texttt{score}_{d}$ of the scoring rules that is nearly optimal in expectation over $\mathcal{D}$. More formally, let cost be an abstract cost function that takes as input a problem instance $Q$ and a scoring rule score and returns some measure of the quality of B&B using score on input $Q$. For example, $\texttt{cost}(Q,\texttt{score})$ might be the number of nodes produced by running B&B using score on input $Q$. We say that an algorithm $(\epsilon,\delta)$-learns a convex combination of the $d$ scoring rules $\texttt{score}_{1},\dots,\texttt{score}_{d}$ if for any distribution $\mathcal{D}$, with probability at least $1-\delta$ over the draw of a sample $\left\{Q_{1},\dots,Q_{m}\right\}\sim\mathcal{D}^{m}$, the algorithm returns a convex combination $\texttt{score}=\hat{\mu}_{1}\texttt{score}_{1}+\cdots+\hat{\mu}_{d}\texttt{score}_{d}$ such that \mathbb{E}_{Q\sim\mathcal{D}}\bigl{[}\texttt{cost}(Q,\texttt{score})\bigr{]}-\mathbb{E}_{Q\sim\mathcal{D}}\bigl{[}\texttt{cost}(Q,\texttt{score}^{*})\bigr{]}\leq\epsilon, where $\texttt{score}^{*}$ is the convex combination of $\texttt{score}_{1},\dots,\texttt{score}_{d}$ with minimal expected cost. In this work, we prove that only a small number of samples is sufficient to ensure $(\epsilon,\delta)$-learnability.

Following prior work (e.g., (Hutter et al., 2009; Kleinberg et al., 2017)), we assume that there is some cap $\kappa$ on the range of the cost function cost. For example, if cost is the size of the search tree, we may choose to terminate the algorithm when the tree size grows beyond some bound $\kappa$. We also assume that the problem instances in the support of $\mathcal{D}$ are over $n$ binary variables, for some $n\in\mathbb{N}$.

Our results hold for cost functions that are tree-constant, which means that for any problem instance $Q$, so long as the scoring rules $\texttt{score}_{1}$ and $\texttt{score}_{2}$ result in the same search tree, $\texttt{cost}(Q,\texttt{score}_{1})=\texttt{cost}(Q,\texttt{score}_{2})$. For example, the size of the search tree is tree-constant.

3.2 Impossibility results for data-independent approaches

In this section, we focus on MILP tree search and prove that it is impossible to find a nearly optimal B&B configuration using a data-independent discretization of the parameters. Specifically, suppose $\texttt{score}_{1}(\mathcal{T},Q,i)=\min\left\{\breve{c}_{Q}-\breve{c}_{Q_{i}^{+}},\breve{c}_{Q}-\breve{c}_{Q_{i}^{-}}\right\}$ and $\texttt{score}_{2}(\mathcal{T},Q,i)=\max\left\{\breve{c}_{Q}-\breve{c}_{Q_{i}^{+}},\breve{c}_{Q}-\breve{c}_{Q_{i}^{-}}\right\}$ and suppose the cost function $\texttt{cost}\left(Q,\mu\texttt{score}_{1}+(1-\mu)\texttt{score}_{2}\right)$ measures the size of the tree produced by B&B when using a fixed but arbitrary node selection policy. We would like to learn a nearly optimal convex combination $\mu\texttt{score}_{1}+(1-\mu)\texttt{score}_{2}$ of these two rules with respect to cost. Gauthier and Ribière (1977) proposed setting $\mu=1/2$, Bénichou et al. (1971) and Beale (1979) suggested setting $\mu=1$, and Linderoth and Savelsbergh (1999) found that $\mu=2/3$ performs well. Achterberg (2009) found that experimentally, $\mu=5/6$ performed best when comparing among $\mu\in\left\{0,1/2,2/3,5/6,1\right\}$.

We show that for any discretization of the parameter space [0, 1], there exists an infinite family of distributions over MILP problem instances such that for any parameter in the discretization, the expected tree size is exponential in $n$. Yet, there exists an infinite number of parameters such that the tree size is just a constant (with probability 1). The proof is in Appendix C.

Theorem 3.1**.**

Let

[TABLE]

and $\texttt{cost}(Q,\mu\texttt{score}_{1}+(1-\mu)\texttt{score}_{2})$ be the size of the tree produced by B&B. For every $a,b$ such that $\frac{1}{3}<a<b<\frac{1}{2}$ and for all even $n\geq 6$, there exists an infinite family of distributions $\mathcal{D}$ over MILP instances with $n$ variables such that if $\mu\in[0,1]\setminus(a,b)$, then

[TABLE]

and if $\mu\in(a,b)$, then with probability 1, $\texttt{cost}\left(Q,\mu\texttt{score}_{1}+(1-\mu)\texttt{score}_{2}\right)=O(1).$ This holds no matter which node selection policy B&B uses.

Proof sketch.

We populate the support of the distribution $\mathcal{D}$ by relying on two helpful theorems: Theorem 3.2 and C.4. In Theorem 3.2, we prove that for all $\mu^{*}\in(0,1)$, there exists an infinite family $\mathcal{F}_{n,\mu^{*}}$ of MILP instances such that for any $Q\in\mathcal{F}_{n,\mu^{*}}$, if $\mu\in\left[0,\mu^{*}\right)$, then the scoring rule $\mu\texttt{score}_{1}+(1-\mu)\texttt{score}_{2}$ results in a B&B tree with $O(1)$ nodes and if $\mu\in\left(\mu^{*},1\right]$, the scoring rule results a tree with $2^{(n-4)/2}$ nodes. Conversely, in Theorem C.4, we prove that there exists an infinite family $\mathcal{G}_{n,\mu^{*}}$ of MILP instances such that for any $Q\in\mathcal{G}_{n,\mu^{*}}$, if $\mu\in\left[0,\mu^{*}\right)$, then the scoring rule $\mu\texttt{score}_{1}+(1-\mu)\texttt{score}_{2}$ results in a B&B tree with $2^{(n-5)/4}$ nodes and if $\mu\in\left(\mu^{*},1\right]$, the scoring rule results a tree with $O(1)$ nodes. Now, let $Q_{a}$ be an arbitrary instance in $\mathcal{G}_{n,a}$ and let $Q_{b}$ be an arbitrary instance in $\mathcal{F}_{n,b}$. The theorem follows by letting $\mathcal{D}$ be a distribution such that $\Pr_{Q\sim\mathcal{D}}\left[Q=Q_{a}\right]=\Pr_{Q\sim\mathcal{D}}\left[Q=Q_{b}\right]=1/2.$ See Figure 2 for an illustration.

Throughout the proof of this theorem, we assume the node selection policy is depth-first search. We then prove that for any infeasible MILP, if NSP and NSP’ are two node selection policies and $\texttt{score}=\mu\texttt{score}_{1}+(1-\mu)\texttt{score}_{2}$ for any $\mu\in[0,1]$, then tree $\mathcal{T}$ B&B builds using NSP and score equals the tree $\mathcal{T}^{\prime}$ it builds using NSP’ and score (see Theorem C.9). Thus, the theorem holds for any node selection policy. ∎

We now provide a proof sketch of Theorem 3.2, which helps us populate the support of the worst-case distributions in Theorem 3.1. The full proof is in Appendix C.

Theorem 3.2**.**

Let

[TABLE]

and $\texttt{cost}(Q,\mu\texttt{score}_{1}+(1-\mu)\texttt{score}_{2})$ be the size of the tree produced by B&B. For all even $n\geq 6$ and all $\mu^{*}\in\left(\frac{1}{3},\frac{1}{2}\right)$, there exists an infinite family $\mathcal{F}_{n,\mu^{*}}$ of MILP instances such that for any $Q\in\mathcal{F}_{n,\mu^{*}}$, if $\mu\in\left[0,\mu^{*}\right)$, then the scoring rule $\mu\texttt{score}_{1}+(1-\mu)\texttt{score}_{2}$ results in a B&B tree with $O(1)$ nodes and if $\mu\in\left(\mu^{*},1\right]$, the scoring rule results a tree with $2^{(n-4)/2}$ nodes.

Proof sketch.

The MILP instances in $\mathcal{F}_{n,\mu^{*}}$ are inspired by a worst-case B&B instance introduced by Jeroslow (1974). He proved that for any odd $n^{\prime}$, every B&B algorithm will build a tree with $2^{(n^{\prime}-1)/2}$ nodes before it determines that for any $\bm{c}\in\mathbb{R}^{n^{\prime}}$, the following MILP is infeasible:

[TABLE]

We build off of this MILP to create the infinite family $\mathcal{F}_{n,\mu^{*}}$. Each MILP in $\mathcal{F}_{n,\mu^{*}}$ combines a hard version of Jeroslow’s instance on $n-3$ variables $\{x_{1},\dots,x_{n-3}\}$ and an easy version on 3 variables $\{x_{n-2},x_{n-1},x_{n}\}$. Branch-and-bound only needs to determine that one of these problems is infeasible in order to terminate. The key idea of this proof is that if B&B branches on all variables in $\{x_{n-2},x_{n-1},x_{n}\}$ first, it will terminate upon making a small tree. However, if B&B branches on all variables in $\{x_{1},\dots,x_{n-3}\}$ first, it will create a tree with exponential size before it terminates. The challenge is to design an objective function that enforces the first behavior when $\mu<\mu^{*}$ and the second behavior when $\mu>\mu^{*}$. Proving this is the bulk of the work.

In a bit more detail, every instance in $\mathcal{F}_{n,\mu^{*}}$ is defined as follows. For any constant $\gamma\geq 1$, let $\bm{c}_{1}=\gamma(1,2,\dots,n-3)$ and let $\bm{c}_{2}=\gamma\left(0,\frac{3}{2},3-\frac{1}{2\mu^{*}}\right)$. Let $\bm{c}=\left(\bm{c}_{1},\bm{c}_{2}\right)\in\mathbb{R}^{n}$ be the concatenation of $\bm{c}_{1}$ and $\bm{c}_{2}$. Let $Q_{\gamma,n}$ be the MILP

[TABLE]

We define $\mathcal{F}_{n,\mu^{*}}=\left\{Q_{n,\gamma}:\gamma\geq 1\right\}.$

For example, if $\gamma=1$ and $n=8$, then $Q_{\gamma,n}$ is

[TABLE]

As is illustrated in Figure 3, we have essentially “glued together” two disjoint versions of Jeroslow’s instance: the first five variables of $Q_{1,8}$ correspond to a “big” version of Jeroslow’s instance and the last three variables correspond to a small version. Since the goal is maximization, the solution to the LP relaxation of $Q_{1,8}$ will try to obtain as much value from the first five variables $\{x_{1},\dots,x_{5}\}$ as it can, but it is constrained to ensure that $2(x_{1}+\cdots+x_{5})=5$. Therefore, the first five variables will be set to $\left(0,0,\frac{1}{2},1,1\right)$. Similarly, the solution to the LP relaxation of $Q_{1,8}$ will set $(x_{6},x_{7},x_{8})=\left(0,\frac{1}{2},1\right)$ because $\frac{3}{2}<3-\frac{1}{2\mu^{*}}$ under our assumption that $\mu^{*}>\frac{1}{3}$. Thus, the solution to the LP relaxation of $Q_{1,8}$ is $\left(0,0,\frac{1}{2},1,1,0,\frac{1}{2},1\right)$. There are only two fractional variables that B&B might branch on: $x_{3}$ and $x_{7}$. Straightforward calculations show that if $\mathcal{T}$ is the B&B tree so far, which just consists of the root node, $\mu\texttt{score}_{1}\left(\mathcal{T},Q_{\gamma,n},3\right)+(1-\mu)\texttt{score}_{2}\left(\mathcal{T},Q_{\gamma,n},3\right)=\frac{\gamma}{2}$ and $\mu\texttt{score}_{1}\left(\mathcal{T},Q_{\gamma,n},7\right)+(1-\mu)\texttt{score}_{2}\left(\mathcal{T},Q_{\gamma,n},7\right)=\frac{3\gamma}{4}-\frac{\mu\gamma}{4\mu^{*}}.$ This means that B&B will branch first on variable $x_{7}$, which corresponds to the small version of Jeroslow’s instance (see Figure 3(b)) if and only if $\frac{\gamma}{2}<\frac{3\gamma}{4}-\frac{\mu\gamma}{4\mu^{*}}$, which occurs if and only if $\mu<\mu^{*}$. We show that this first branch sets off a cascade: if B&B branches first on variable $x_{7}$, then it will proceed to branch on all variables in $\{x_{6},x_{7},x_{8}\}$, thus terminating upon making a small tree. Meanwhile, if it branches on variable $x_{3}$ first, it will then only branch on variables in $\{x_{1},\dots,x_{5}\}$, creating a larger tree.

In the full proof, we generalize beyond eight variables to $n$, and expand the large version of Jeroslow’s instance (as depicted in Figure 3(a)) from five variables to $n-3$. When $\mu<\mu^{*}$, we simply track B&B’s progress to make sure it only branches on variables from the small version of Jeroslow’s instance ($x_{n-2},x_{n-1},x_{n}$) before figuring out the MILP is infeasible. Therefore, the tree will have constant size. When $\mu>\mu^{*}$, we prove by induction that if B&B has only branched on variables from the big version of Jeroslow’s instance ($x_{1},\dots,x_{n-3}$), it will continue to only branch on those variables. We also prove it will branch on about half of these variables along each path of the B&B tree. The tree will thus have exponential size. ∎

3.3 Sample complexity guarantees

We now provide worst-case guarantees on the number of samples sufficient to $(\epsilon,\delta)$-learn a convex combination of scoring rules. These results bound the number of samples sufficient to ensure that for any convex combination $\texttt{score}=\mu_{1}\texttt{score}_{1}+\cdots+\mu_{d}\texttt{score}_{d}$ of scoring rules, the empirical cost of tree search using score is close to its expected cost. Therefore, the algorithm designer can optimize $\left(\mu_{1},\dots,\mu_{d}\right)$ over the samples without any further knowledge of the distribution. Moreover these sample complexity guarantees apply for any procedure the algorithm designer uses to tune $\left(\mu_{1},\dots,\mu_{d}\right)$, be it an approximation, heuristic, or optimal algorithm. She can use our guarantees to bound the number of samples she needs to ensure that performance on the sample generalizes to the distribution.

In Section 3.3.1 we provide generalization guarantees for a family of scoring rules we call path-wise, which includes many well-known scoring rules as special cases. In this case, the number of samples is surprisingly small given the complexity of these problems: it grows only quadratically with the number of variables. In Section 3.3.2, we provide guarantees that apply to any scoring rule, path-wise or otherwise.

3.3.1 Path-wise scoring rules

The guarantees in this section apply broadly to a class of scoring rules we call path-wise scoring rules. Given a node $Q$ in a search tree $\mathcal{T}$, we denote the path from the root of $\mathcal{T}$ to the node $Q$ as $\mathcal{T}_{Q}$. The path $\mathcal{T}_{Q}$ includes all nodes and edge labels from the root of $\mathcal{T}$ to $Q$. For example, Figure 4(b) illustrates the path $\mathcal{T}_{Q}$ from the root of the tree $\mathcal{T}$ in Figure 4(a) to the node labeled $Q$. We now state the definition of path-wise scoring rules.

Definition 3.1** (Path-wise scoring rule).**

The function score is a path-wise scoring rule if for all search trees $\mathcal{T}$, all nodes $Q$ in $\mathcal{T}$, and all variables $x_{i}$,

[TABLE]

where $\mathcal{T}_{Q}$ is the path from the root of $\mathcal{T}$ to $Q$.222Under this definition, the scoring rule can simulate B&B for any number of steps starting at any point in the tree and use that information to calculate the score, so long as Equality (2) always holds. See Figure 4 for an illustration.

Definition 3.1 requires that if the node $Q$ appears at the end of the same path in two different B&B trees, then any path-wise scoring rule must assign every variable the same score with respect to $Q$ in both trees.

Path-wise scoring rules include many well-studied rules as special cases, such as the most fractional, product, and linear scoring rules, as defined in Section 2.1.2. The same is true when B&B only partially solves the LP relaxations of $Q_{i}^{-}$ and $Q_{i}^{+}$ for every variable $x_{i}$ by running a small number of simplex iterations, as we describe in Section 2.1.2 and as is our approach in our experiments. In fact, these scoring rules depend only on the node in question, rather than the path from the root to the node. We present our sample complexity bound for the more general class of path-wise scoring rules because this class captures the level of generality the proof holds for. On the other hand, pseudo-cost branching (Bénichou et al., 1971; Gauthier and Ribière, 1977; Linderoth and Savelsbergh, 1999) and reliability branching (Achterberg et al., 2005), two widely-used branching strategies, are not path-wise, but our more general results from Section 3.3.2 do apply to those strategies.

In order to prove our generalization guarantees, we make use of the following key structure which bounds the number of search trees branch-and-bound will build on a given instance over the entire range of parameters. In essence, this is a bound on the intrinsic complexity of the algorithm class defined by the range of parameters, and this bound on algorithm class’s intrinsic complexity implies strong generalization guarantees.

Lemma 3.3**.**

Let cost be a tree-constant cost function, let $\texttt{score}_{1}$ and $\texttt{score}_{2}$ be two path-wise scoring rules, and let $Q$ be an arbitrary problem instance over $n$ binary variables. There are $T\leq 2^{n(n-1)/2}n^{n}$ intervals $I_{1},\dots,I_{T}$ partitioning $[0,1]$ where for any interval $I_{j}$, across all $\mu\in I_{j}$, the scoring rule $\mu\texttt{score}_{1}+(1-\mu)\texttt{score}_{2}$ results in the same search tree.

Proof.

We prove this lemma first by considering the actions of an alternative algorithm $A^{\prime}$ which runs exactly like B&B, except it only fathoms nodes if they are integral or infeasible. We then relate the behavior of $A^{\prime}$ to the behavior of B&B to prove the lemma.

First, we prove the following bound on the number of search trees $A^{\prime}$ will build on a given instance over the entire range of parameters. This bound matches that in the lemma statement.

Claim 3.4**.**

There are $T\leq 2^{n(n-1)/2}n^{n}$ intervals $I_{1},\dots,I_{T}$ partitioning $[0,1]$ where for any interval $I_{j}$, the search tree $A^{\prime}$ builds using the scoring rule $\mu\texttt{score}_{1}+(1-\mu)\texttt{score}_{2}$ is invariant across all $\mu\in I_{j}$.333This claim holds even when $\texttt{score}_{1}$ and $\texttt{score}_{2}$ are members of the more general class of depth-wise scoring rules, which we define as follows. For any search tree $\mathcal{T}$ of depth $\text{depth}(\mathcal{T})$ and any $j\in[n]$, let $\mathcal{T}[j]$ be the subtree of $\mathcal{T}$ consisting of all nodes in $\mathcal{T}$ of depth at most $j$. We say that score is a depth-wise scoring rule if for all search trees $\mathcal{T}$, all $j\in[\text{depth}(\mathcal{T})]$, all nodes $Q$ of depth $j$, and all variables $x_{i}$, $\texttt{score}(\mathcal{T},Q,i)=\texttt{score}(\mathcal{T}[j],Q,i)$.

Proof sketch of Claim 3.4.

We prove this claim by induction. For a tree $\mathcal{T}$, let $\mathcal{T}[i]$ be the nodes of depth at most $i$. We prove that for $i\in\{1,\dots,n\}$, there are $T\leq 2^{i(i-1)/2}n^{i}$ intervals $I_{1},\dots,I_{T}$ partitioning $[0,1]$ where for any interval $I_{j}$ and any two parameters $\mu,\mu^{\prime}\in I_{j}$, if $\mathcal{T}$ and $\mathcal{T}^{\prime}$ are the trees $A^{\prime}$ builds using the scoring rules $\mu\texttt{score}_{1}+(1-\mu)\texttt{score}_{2}$ and $\mu^{\prime}\texttt{score}_{1}+(1-\mu^{\prime})\texttt{score}_{2}$, respectively, then $\mathcal{T}[i]=\mathcal{T}^{\prime}[i]$. Suppose that this is indeed the case for some $i\in\{1,\dots,n\}$ and consider an arbitrary interval $I_{j}$ and any two parameters $\mu,\mu^{\prime}\in I_{j}$. Consider an arbitrary node $Q$ in $\mathcal{T}[i-1]$ (or equivalently, $\mathcal{T}^{\prime}[i-1]$) at depth $i-1$. If $Q$ is integral or infeasible, then it will be fathomed no matter which parameter $\mu\in I_{j}$ the algorithm $A^{\prime}$ uses. Otherwise, for all $\mu\in I_{j}$, let $\mathcal{T}_{\mu}$ be the state of the search tree $A^{\prime}$ builds using the scoring rule $\mu\texttt{score}_{1}+(1-\mu)\texttt{score}_{2}$ at the point when it branches on $Q$. By the inductive hypothesis, we know that across all $\mu\in I_{j}$, the path from the root to $Q$ in $\mathcal{T}_{\mu}$ is invariant, and we refer to this path as $\mathcal{T}_{Q}$. Given a parameter $\mu\in I_{j}$, the variable $x_{k}$ will be branched on at node $Q$ so long as $k=\textnormal{argmax}_{\ell}\left\{\mu\texttt{score}_{1}(\mathcal{T}_{\mu},Q,\ell)+(1-\mu)\texttt{score}_{2}(\mathcal{T}_{\mu},Q,\ell)\right\},$ or equivalently, so long as $k=\textnormal{argmax}_{\ell}\left\{\mu\texttt{score}_{1}(\mathcal{T}_{Q},Q,\ell)+(1-\mu)\texttt{score}_{2}(\mathcal{T}_{Q},Q,\ell)\right\}$. In other words, the decision of which variable to branch on is determined by a convex combination of the constant values $\texttt{score}_{1}(\mathcal{T}_{Q},Q,\ell)$ and $\texttt{score}_{2}(\mathcal{T}_{Q},Q,\ell)$ no matter which parameter $\mu\in I_{j}$ the algorithm $A^{\prime}$ uses. Here, we critically use the fact that the scoring rule is path-wise.

Since $\mu\texttt{score}_{1}(\mathcal{T}_{Q},Q,\ell)+(1-\mu)\texttt{score}_{2}(\mathcal{T}_{Q},Q,\ell)$ is a linear function of $\mu$ for all $\ell$, there are at most $n$ intervals subdividing the interval $I_{j}$ such that the variable branched on at node $Q$ is fixed.

Moreover, there are at most $2^{i-1}$ nodes at depth $i-1$, and each node similarly contributes a subpartition of $I_{j}$ of size $n$. If we merge all $2^{i-1}$ partitions, we have $T^{\prime}\leq 2^{i-1}(n-1)+1$ intervals $I_{1}^{\prime},\dots,I_{T^{\prime}}^{\prime}$ partitioning $I_{j}$ where for any interval $I_{p}^{\prime}$ and any two parameters $\mu,\mu^{\prime}\in I_{p}^{\prime}$, if $\mathcal{T}$ and $\mathcal{T}^{\prime}$ are the trees $A^{\prime}$ builds using the scoring rules $\mu\texttt{score}_{1}+(1-\mu)\texttt{score}_{2}$ and $\mu^{\prime}\texttt{score}_{1}+(1-\mu^{\prime})\texttt{score}_{2}$, respectively, then $\mathcal{T}[i]=\mathcal{T}^{\prime}[i]$. We can similarly subdivide each interval $I_{1},\dots,I_{T}$. The claim then follows from counting the number of subdivisions. ∎

Next, we explicitly relate the behavior of B&B to $A^{\prime}$, proving that the search tree B&B builds is a rooted subtree of the search tree $A^{\prime}$ builds.

Claim 3.5**.**

Given a parameter $\mu\in[0,1]$, let $\mathcal{T}$ and $\mathcal{T}^{\prime}$ be the trees B&B and $A^{\prime}$ build respectively using the scoring rule $\mu\texttt{score}_{1}+(1-\mu)\texttt{score}_{2}$. For any node $Q$ of $\mathcal{T}$, let $\mathcal{T}_{Q}$ be the path from the root of $\mathcal{T}$ to $Q$. Then $\mathcal{T}_{Q}$ is a rooted subtree of $\mathcal{T}^{\prime}$.

Proof of Claim 3.5.

The path $\mathcal{T}_{Q}$ can be labeled by a sequence of indices from $\{0,1\}$ and a sequence of variables from $\{x_{1},\dots,x_{n}\}$ describing which variable is branched on and which value it takes on along the path $\mathcal{T}_{Q}$. Let $((j_{1},x_{i_{1}}),\dots,(j_{t},x_{i_{t}}))$ be this sequence of labels, where $t$ is the number of edges in $\mathcal{T}_{Q}$. We can similarly label every edge in $\mathcal{T}^{\prime}$. We claim that there exists a path beginning at the root of $\mathcal{T}^{\prime}$ with the labels $((j_{1},x_{i_{1}}),\dots,(j_{t},x_{i_{t}}))$.

For a contradiction, suppose no such path exists. Let $(j_{\tau},x_{i_{\tau}})$ be the earliest label in the sequence $((j_{1},x_{i_{1}}),\dots,(j_{t},x_{i_{t}}))$ where there is a path beginning at the root of $\mathcal{T}^{\prime}$ with the labels $((j_{1},x_{i_{1}}),\dots,(j_{\tau-1},x_{i_{\tau-1}}))$, but there is no way to continue the path using an edge labeled $(j_{\tau},x_{i_{\tau}})$. There are exactly two reasons why this could be the case:

The node at the end of the path with labels $((j_{1},x_{i_{1}}),\dots,(j_{\tau-1},x_{i_{\tau-1}}))$ was fathomed by $A^{\prime}$. 2. 2.

The algorithm $A^{\prime}$ branched on a variable other than $x_{i_{\tau}}$ at the end of the path labeled $((j_{1},x_{i_{1}}),\dots,(j_{\tau-1},x_{i_{\tau-1}}))$.

In the first case, since $A^{\prime}$ only fathoms a node if it is integral or infeasible, we know that B&B will also fathom the node at the end of the path with labels $((j_{1},x_{i_{1}}),\dots,(j_{\tau-1},x_{i_{\tau-1}}))$. However, this is not the case since B&B next branches on the variable $x_{i_{\tau}}$.

The second case is also not possible since the scoring rules are both path-wise. In a bit more detail, let $Q^{\prime}$ be the node at the end of the path with labels $((j_{1},x_{i_{1}}),\dots,(j_{\tau-1},x_{i_{\tau-1}}))$. We refer to this path as $\mathcal{T}_{Q^{\prime}}$. Let $\bar{\mathcal{T}}$ (respectively, $\bar{\mathcal{T}}^{\prime}$) be the state of the search tree B&B (respectively, $A$’) has built at the point it branches on $Q^{\prime}$. We know that $\mathcal{T}_{Q^{\prime}}$ is the path from the root to $Q^{\prime}$ in both of the trees $\bar{\mathcal{T}}$ and $\bar{\mathcal{T}}^{\prime}$. Therefore, for all variables $x_{k}$, $\mu\texttt{score}_{1}(\bar{\mathcal{T}},Q^{\prime},k)+(1-\mu)\texttt{score}_{2}(\bar{\mathcal{T}},Q^{\prime},k)=\mu\texttt{score}_{1}(\mathcal{T}_{Q^{\prime}},Q^{\prime},k)+(1-\mu)\texttt{score}_{2}(\mathcal{T}_{Q^{\prime}},Q^{\prime},k)=\mu\texttt{score}_{1}(\bar{\mathcal{T}}^{\prime},Q^{\prime},k)+(1-\mu)\texttt{score}_{2}(\bar{\mathcal{T}}^{\prime},Q^{\prime},k)$. This means that B&B and $A^{\prime}$ will choose the same variable to branch on at the node $Q^{\prime}$.

Therefore, we have reached a contradiction, so the claim holds. ∎

Next, we use Claims 3.4 and 3.5 to prove Lemma 3.3. Let $I_{1},\dots,I_{T}$ be the intervals guaranteed to exist by Claim 3.4 and let $I_{t}$ be an arbitrary one of the intervals. Let $\mu^{\prime}$ and $\mu^{\prime\prime}$ be two arbitrary parameters from $I_{t}$. We will prove that the scoring rules $\mu^{\prime}\texttt{score}_{1}+(1-\mu^{\prime})\texttt{score}_{2}$ and $\mu^{\prime\prime}\texttt{score}_{1}+(1-\mu^{\prime\prime})\texttt{score}_{2}$ result in the same B&B search tree. For a contradiction, suppose that this is not the case. Consider the first iteration where B&B using the scoring rule $\mu^{\prime}\texttt{score}_{1}+(1-\mu^{\prime})\texttt{score}_{2}$ differs from B&B using the scoring rule $\mu^{\prime\prime}\texttt{score}_{1}+(1-\mu^{\prime\prime})\texttt{score}_{2}$. By iteration, we mean lines 5 through 19 of Algorithm 1. Up until this iteration, B&B has built the same partial search tree $\mathcal{T}$. Since the node selection policy does not depend on $\mu^{\prime}$ or $\mu^{\prime\prime}$, B&B will choose the same leaf $Q$ of the B&B search tree to branch on no matter which scoring rule it uses.

Suppose B&B chooses different variables to branch on in Step 6 of Algorithm 1 depending on whether it uses the scoring rule $\mu^{\prime}\texttt{score}_{1}+(1-\mu^{\prime})\texttt{score}_{2}$ or $\mu^{\prime\prime}\texttt{score}_{1}+(1-\mu^{\prime\prime})\texttt{score}_{2}$. Let $\mathcal{T}_{Q}$ be the path from the root of $\mathcal{T}$ to $Q$. By Claim 3.4, we know that the algorithm $A^{\prime}$ builds the same search tree using the two scoring rules. Let $\bar{\mathcal{T}}^{\prime}$ (respectively, $\bar{\mathcal{T}}^{\prime\prime}$) be the state of the search tree $A^{\prime}$ has built using the scoring rule $\mu^{\prime}\texttt{score}_{1}+(1-\mu^{\prime})\texttt{score}_{2}$ (respectively, $\mu^{\prime\prime}\texttt{score}_{1}+(1-\mu^{\prime\prime})\texttt{score}_{2}$) by the time it branches on the node $Q$. By Claims 3.4 and 3.5, we know that $\mathcal{T}_{Q}$ is the path of from the root to $Q$ of both $\bar{\mathcal{T}}^{\prime}$ and $\bar{\mathcal{T}}^{\prime\prime}$. By Claim 3.4, we know that $A^{\prime}$ will branch on the same variable $x_{i}$ at the node $Q$ in both the trees $\bar{\mathcal{T}}^{\prime}$ and $\bar{\mathcal{T}}^{\prime\prime}$, so $i=\textnormal{argmax}_{j}\left\{\mu^{\prime}\texttt{score}_{1}(\bar{\mathcal{T}}^{\prime},Q,j)+(1-\mu^{\prime})\texttt{score}_{2}(\bar{\mathcal{T}}^{\prime},Q,j)\right\}$, or equivalently,

[TABLE]

and $i=\textnormal{argmax}_{j}\left\{\mu^{\prime\prime}\texttt{score}_{1}(\bar{\mathcal{T}}^{\prime\prime},Q,j)+(1-\mu^{\prime\prime})\texttt{score}_{2}(\bar{\mathcal{T}}^{\prime\prime},Q,j)\right\}$, or equivalently,

[TABLE]

Returning to the search tree $\mathcal{T}$ that B&B is building, Equation (3) implies that

[TABLE]

and Equation (4) implies that $i=\textnormal{argmax}_{j}\left\{\mu^{\prime\prime}\texttt{score}_{1}(\mathcal{T},Q,j)+(1-\mu^{\prime\prime})\texttt{score}_{2}(\mathcal{T},Q,j)\right\}$. Therefore, B&B will branch on $x_{i}$ at the node $Q$ no matter which scoring rule it uses.

Finally, since the trees B&B has built so far are identical, the choice of whether or not to fathom the children $Q_{i}^{+}$ and $Q_{i}^{-}$ does not depend on the scoring rule, so B&B will fathom the same nodes no matter whether it uses the scoring rule $\mu^{\prime}\texttt{score}_{1}+(1-\mu^{\prime})\texttt{score}_{2}$ or $\mu^{\prime\prime}\texttt{score}_{1}+(1-\mu^{\prime\prime})\texttt{score}_{2}$. Therefore, we have reached a contradiction: the iterations were identical. We conclude that the lemma holds. ∎

We now show how this structure implies a generalization guarantee. We formulate our guarantees in terms of pseudo-dimension. By classic results from learning theory, pseudo-dimension immediately implies generalization guarantees. Pseudo-dimension is defined as follows.

Definition 3.2** (Pseudo-dimension (Pollard, 1984)).**

Let $\mathcal{F}$ be a class of functions mapping an abstract domain $\mathcal{Z}$ to the set $[-\kappa,\kappa]$. Let $\mathcal{S}=\left\{z_{1},\dots,z_{m}\right\}$ be a subset of $\mathcal{Z}$ and let $r_{1},\dots,r_{m}\in\mathbb{R}$ be a set of targets. We say that $r_{1},\dots,r_{m}$ witness the shattering of $\mathcal{S}$ by $\mathcal{F}$ if for all $\mathcal{S}^{\prime}\subseteq\mathcal{S}$, there exists some function $f_{\mathcal{S}^{\prime}}\in\mathcal{F}$ such that for all $z_{i}\in{\mathcal{S}^{\prime}}$, $f_{\mathcal{S}^{\prime}}\left(z_{i}\right)\leq r_{i}$ and for all $z_{i}\not\in{\mathcal{S}^{\prime}}$, $f_{\mathcal{S}^{\prime}}\left(z_{i}\right)>r_{i}$. If there exists some $\bm{r}\in\mathbb{R}^{m}$ that witnesses the shattering of $\mathcal{S}$ by $\mathcal{F}$, then we say that $\mathcal{S}$ is shatterable by $\mathcal{F}$. Finally, the pseudo-dimension of $\mathcal{F}$, denoted $\textnormal{Pdim}\left(\mathcal{F}\right)$, is the size of the largest set that is shatterable by $\mathcal{F}$.

Theorem 3.6 provides generalization bounds in terms of pseudo-dimension.

Theorem 3.6** (Pollard (1984)).**

For any distribution $\mathcal{D}$ over $\mathcal{Z}$, with probability at least $1-\delta$ over the draw of $\mathcal{S}=\left\{z_{1},\dots,z_{m}\right\}\sim\mathcal{D}^{m}$, for all $f\in\mathcal{F}$,

[TABLE]

Theorem 3.7**.**

Let cost be a tree-constant cost function, let $\texttt{score}_{1}$ and $\texttt{score}_{2}$ be two path-wise scoring rules, and let $\mathcal{C}$ be the set of functions $\left\{\texttt{cost}\left(\cdot,\mu\texttt{score}_{1}+(1-\mu)\texttt{score}_{2}\right):\mu\in[0,1]\right\}$. Then $\textnormal{Pdim}(\mathcal{C})=O\left(n^{2}\right).$

Proof.

Suppose that $\textnormal{Pdim}(\mathcal{C})=m$ and let $\mathcal{S}=\left\{Q_{1},\dots,Q_{m}\right\}$ be a shatterable set of problem instances. We know there exists a set of targets $r_{1},\dots,r_{m}\in\mathbb{R}$ that witness the shattering of $\mathcal{S}$ by $\mathcal{C}$. This means that for every $\mathcal{S}^{\prime}\subseteq\mathcal{S}$, there exists a parameter $\mu_{\mathcal{S}^{\prime}}$ such that if $Q_{i}\in\mathcal{S}$, then $\texttt{cost}\left(Q_{i},\mu_{\mathcal{S}^{\prime}}\texttt{score}_{1}+\left(1-\mu_{\mathcal{S}^{\prime}}\right)\texttt{score}_{2}\right)\leq r_{i}$. Otherwise $\texttt{cost}\left(Q_{i},\mu_{\mathcal{S}^{\prime}}\texttt{score}_{1}+\left(1-\mu_{\mathcal{S}^{\prime}}\right)\texttt{score}_{2}\right)>r_{i}$. Let $M=\left\{\mu_{\mathcal{S}^{\prime}}:\mathcal{S}^{\prime}\subseteq\mathcal{S}\right\}$. We will prove that $|M|\leq m2^{n(n-1)/2}n^{n}+1$, and since $2^{m}=|M|$, this means that $\textnormal{Pdim}(\mathcal{C})=m=O\left(\log\left(2^{n(n-1)/2}n^{n}\right)\right)=O\left(n^{2}\right)$ (see Lemma C.10 in Appendix C).

To prove that $|M|\leq m2^{n(n-1)/2}n^{n}+1$, we rely on Lemma 3.3, which tells us that for any problem instance $Q$, there are $T\leq 2^{n(n-1)/2}n^{n}$ intervals $I_{1},\dots,I_{T}$ partitioning $[0,1]$ where for any interval $I_{j}$, across all $\mu\in I_{j}$, the scoring rule $\mu\texttt{score}_{1}+(1-\mu)\texttt{score}_{2}$ results in the same search tree. If we merge all $T$ intervals for all samples in $\mathcal{S}$, we are left with $T^{\prime}\leq m2^{n(n-1)/2}n^{n}+1$ intervals $I_{1}^{\prime},\dots,I_{T^{\prime}}^{\prime}$ where for any interval $I_{j}^{\prime}$ and any $Q_{i}\in\mathcal{S}$, $\texttt{cost}\left(Q_{i},\mu\texttt{score}_{1}+\left(1-\mu\right)\texttt{score}_{2}\right)$ is constant for all $\mu\in I_{j}^{\prime}$. Therefore, at most one element of $M$ can come from each interval, meaning that $|M|\leq T^{\prime}\leq m2^{n(n-1)/2}n^{n}+1$, as claimed. ∎

3.3.2 General scoring rules

In this section, we provide generalization guarantees that apply to learning convex combinations of any set of scoring rules. Unlike the guarantees in Section 3.3.1, they depend on the size of the search trees B&B is allowed to build. For example, we may choose to terminate the algorithm when the tree size grows beyond some bound $\bar{\kappa}$. The following lemma corresponds to Lemma 3.3 for this setting. For the full proof, see Lemma E.4 in Appendix E.2.1 which proves the lemma for a more general tree search algorithm.

Lemma 3.8**.**

Let cost be a tree-constant cost function, let $\texttt{score}_{1},\dots,\texttt{score}_{d}$ be $d$ arbitrary scoring rules, and let $Q$ be an arbitrary MILP over $n$ binary variables. Suppose we limit B&B to producing search trees of size $\bar{\kappa}$. There is a set $\mathcal{H}$ of at most $n^{2(\bar{\kappa}+1)}$ hyperplanes such that for any connected component $R$ of $[0,1]^{d}\setminus\mathcal{H}$, the search tree B&B builds using the scoring rule $\mu_{1}\texttt{score}_{1}+\cdots+\mu_{d}\texttt{score}_{d}$ is invariant across all $(\mu_{1},\dots,\mu_{d})\in R$.

Proof sketch.

The proof has two steps. In Claim E.5, we show that there are at most $n^{\bar{\kappa}}$ different search trees that B&B might produce for the instance $Q$ as we vary the mixing parameter vector $(\mu_{1},\dots,\mu_{d})$. In Claim E.6, for each of the possible search trees $\mathcal{T}$ that might be produced, we show that the set of parameter values $(\mu_{1},\dots,\mu_{d})$ which give rise to that tree lie in the intersection of $n^{\bar{\kappa}+2}$ halfspaces. Of course, each of these halfspaces is defined by a hyperplane. Let $\mathcal{H}$ be the union of all $n^{\bar{\kappa}+2}$ hyperplanes over all $n^{\bar{\kappa}}$ trees. We know that for any connected component $R$ of $[0,1]^{d}\setminus\mathcal{H}$, the search tree B&B builds using the scoring rule $\mu_{1}\texttt{score}_{1}+\cdots+\mu_{d}\texttt{score}_{d}$ is invariant across all $(\mu_{1},\dots,\mu_{d})\in R$, so the lemma statement holds. ∎

In the same way Lemma 3.3 implies the pseudo-dimension bound in Theorem 3.7, Lemma 3.8 also implies a pseudo-dimension bound. The proof is similar to that of Theorem 3.7. See Theorem E.7 in Appendix E.2.1 which proves the theorem for a more general tree search algorithm.

Theorem 3.9**.**

Let cost be a tree-constant cost function and let $\texttt{score}_{1},\dots,\texttt{score}_{d}$ be $d$ arbitrary scoring rules. Suppose we limit B&B to producing search trees of size $\bar{\kappa}$. Let $\mathcal{C}$ be the set of functions $\left\{\texttt{cost}\left(\cdot,\mu\texttt{score}_{1}+\cdots+\mu_{d}\texttt{score}_{d}\right):(\mu_{1},\dots,\mu_{d})\in[0,1]^{d}\right\}$. Then $\textnormal{Pdim}(\mathcal{C})=O\left(d\bar{\kappa}\log n+d\log d\right).$

Naturally, if the function cost measures the size of the search tree B&B returns capped at some value $\kappa$ as described in Section 3.1, we obtain the following corollary.

Corollary 3.10**.**

Let cost be a tree-constant cost function, let $\texttt{score}_{1},\dots,\texttt{score}_{d}$ be $d$ arbitrary scoring rules, and let $\kappa\in\mathbb{N}$ be an arbitrary tree size bound. Suppose that for any problem instance $Q$, $\texttt{cost}(Q,\mu\texttt{score}_{1}+\cdots+\mu_{d}\texttt{score}_{d})$ equals the minimum of the following two values: 1) The number of nodes in the search tree B&B builds using the scoring rule $\mu\texttt{score}_{1}+\cdots+\mu_{d}\texttt{score}_{d}$ on input $Q$; and 2) $\kappa$. For any distribution $\mathcal{D}$ over problem instances $Q$ with at most $n$ variables, with probability at least $1-\delta$ over the draw $\{Q_{1},\dots,Q_{m}\}\sim\mathcal{D}^{m},$ for any $\mu\in[0,1]$,

[TABLE]

3.3.3 Learning algorithm

For the case where we wish to learn the optimal tradeoff between two scoring rules, we provide an algorithm in Appendix C that finds the empirically optimal parameter $\hat{\mu}$ given a sample of $m$ problem instances. By Theorems 3.7, 3.9, and 3.6, we know that so long as $m$ is sufficiently large, $\hat{\mu}$ is nearly optimal in expectation. Our algorithm stems from the observation that for any tree-constant cost function cost and any problem instance $Q$, $\texttt{cost}\left(Q,\mu\texttt{score}_{1}+(1-\mu)\texttt{score}_{2}\right)$ is simple: it is a piecewise-constant function of $\mu$ with a finite number of pieces. This is the same observation that we prove in Lemma 3.8. Given a sample of problem instances $\mathcal{S}=\left\{Q_{1},\dots,Q_{m}\right\}$, our ERM algorithm constructs all $m$ piecewise-constant functions, takes their average, and finds the minimizer of that function. In practice, we find that the number of pieces making up this piecewise-constant function is small, so our algorithm can learn over a training set of many problem instances.

4 Experiments

In this section, we show that the parameter of the variable selection rule in B&B algorithms for MILP can have a dramatic effect on the average tree size generated for several domains, and no parameter value is effective across the multiple natural distributions.

Experimental setup.

We use the C API of IBM ILOG CPLEX 12.8.0.0 to override the default variable selection rule using a branch callback. Additionally, our callback performs extra book-keeping to determine a finite set of values for the parameter that give rise to all possible B&B trees for a given instance (for the given choice of branching rules that our algorithm is learning to weight). This ensures that there are no good or bad values for the parameter that get skipped; such skipping could be problematic according to our theory in Section 3.2. We run CPLEX exactly once for each possible B&B tree on each instance. Following Khalil et al. (2016); Fischetti and Monaci (2012), and Karzan et al. (2009), we disable CPLEX’s cuts and primal heuristics, and we also disable its root-node preprocessing. The CPLEX node selection policy is set to “best bound” (aka. $A^{*}$ in AI), which is the most typical choice in MILP. All experiments are run on a cluster of 64 c3.large Amazon AWS instances.

For each of the following application domains, Figure 6 shows the average B&B tree size produced for each possible value of the $\mu$ parameter for the linear scoring rule, averaged over $m$ independent samples from the distribution.

Combinatorial auctions.

We generate $m=100$ instances of the combinatorial auction winner determination problem under the OR-bidding language (Sandholm, 2002), which makes this problem equivalent to weighted set packing. The problem is NP-complete. We encode each instance as a binary MILP (see Example 2.1). We use the Combinatorial Auction Test Suite (CATS) (Leyton-Brown et al., 2000) to generate these instances. We use the “arbitrary” generator with 200 bids and 100 goods and “regions” generator with 400 bids and 200 goods.

Facility location.

Suppose there is a set $I$ of customers and a set $J$ of facilities that have not yet been built. The facilities each produce the same good, and each consumer demands one unit of that good. Consumer $i$ can obtain some fraction $y_{ij}$ of the good from facility $j$, which costs them $d_{ij}y_{ij}$. Moreover, it costs $f_{j}$ to construct facility $j$. The goal is to choose a subset of facilities to construct while minimizing total cost. In Appendix D we show how to formulate facility location as a MILP. We generate $m=500$ instances with 70 facilities and 70 customers each. Each cost $d_{ij}$ is uniformly sampled from $\left[0,10^{4}\right]$ and each cost $f_{j}$ is uniformly sampled from $\left[0,3\cdot 10^{3}\right]$. This distribution has regularly been used to generate benchmark sets for facility location (Gilpin and Sandholm, 2011; Alekseeva et al., 2015; Goldengorin et al., 2011; Kochetov and Ivanenko, 2005; Homberger and Gehring, 2008).

Clustering.

Given $n$ points $P=\left\{p_{1},\dots,p_{n}\right\}$ and pairwise distances $d(p_{i},p_{j})$ between each pair of points $p_{i}$ and $p_{j}$, the goal of $k$-means clustering is to find $k$ centers $C=\left\{c_{1},\dots,c_{k}\right\}\subseteq P$ such that the following objective function is minimized: $\sum_{i=1}^{n}\min_{j\in[k]}d\left(p_{i},c_{j}\right)^{2}.$ In Appendix D, we show how to formulate this problem as a MILP. We generate $m=500$ instances with 35 points each and $k=5$. We set $d(i,i)=0$ for all $i$ and choose $d(i,j)$ uniformly at random from $[0,1]$ for $i\neq j$. These distances do not satisfy the triangle inequality and they are not symmetric (i.e., $d(i,j)\neq d(j,i)$), which tends to lead to harder MILP instances than using Euclidean distances between randomly chosen points in $\mathbb{R}^{d}$.

Agnostically learning linear separators.

Let $\bm{p}_{1},\dots,\bm{p}_{N}$ be $N$ points in $\mathbb{R}^{d}$ labeled by $z_{1},\dots,z_{N}\in\{-1,1\}$. Suppose we wish to learn a linear separator $\bm{w}\in\mathbb{R}^{d}$ that minimizes 0-1 loss, i.e., $\sum_{i=1}^{N}\mathbf{1}_{\left\{z_{i}\left\langle\bm{p}_{i},\bm{w}\right\rangle<0\right\}}$. In Appendix D, we show how to formulate this problem as a MILP. We generate $m=500$ problem instances with 50 points $\bm{p}_{1},\dots,\bm{p}_{50}$ from the 2-dimensional standard normal distribution. We sample the true linear separator $\bm{w}^{*}$ from the 2-dimensional standard Gaussian distribution and label point $\bm{p}_{i}$ by $z_{i}=\textnormal{sign}(\langle\bm{w}^{*},\bm{p}_{i}\rangle)$. We then choose $10$ random points and flip their labels so that there is no consistent linear separator.

Experimental results.

The relationship between the variable selection parameter and the average tree size varies greatly from application to application. This implies that the parameters should be tuned on a per-application basis, and that no parameter value is universally effective. In particular, the optimal parameter for the “regions” combinatorial auction problem, facility location, and clustering is close to 1. However, that value is severely suboptimal for the “arbitrary” combinatorial auction domain, resulting in trees that are three times the size of the trees obtained under the optimal parameter value.

Data-dependent guarantees.

We now explore data-dependent generalization guarantees. To prove our worst-case guarantee Theorem 3.7, we show in Lemma 3.3 that for any MILP instance over $n$ binary variables, there are $T\leq 2^{n(n-1)/2}n^{n}$ intervals $I_{1},\dots,I_{T}$ partitioning $[0,1]$ where for any interval $I_{j}$, across all $\mu\in I_{j}$, the scoring rule $\mu\texttt{score}_{1}+(1-\mu)\texttt{score}_{2}$ results in the same B&B tree. Our generalization guarantees grow logarithmically in the number of intervals. In practice, we find that the number of intervals partitioning $[0,1]$ is much smaller than $2^{n(n-1)/2}n^{n}$. In this section, we take advantage of this data-dependent simplicity to derive stronger generalization guarantees when the number of intervals partitioning $[0,1]$ is small. To do so, we move from pseudo-dimension to Rademacher complexity (Bartlett and Mendelson, 2002; Koltchinskii, 2001) since Rademacher complexity implies distribution-dependent guarantees whereas pseudo-dimension implies generalization guarantees that are worst-case over the distribution. We now define empirical Rademacher complexity.

Definition 4.1** (Empirical Rademacher complexity).**

Let $\mathcal{F}$ be a class of functions mapping an abstract domain $\mathcal{Z}$ to the set $[-\kappa,\kappa]$. The empirical Rademacher complexity of $\mathcal{F}$ with respect to a sample $\mathcal{S}=\left\{z_{1},\dots,z_{m}\right\}\subseteq\mathcal{Z}$ of size $m$ is defined as

[TABLE]

The following generalization guarantee based on Rademacher complexity is well-known.

Theorem 4.1**.**

For any distribution $\mathcal{D}$ over $\mathcal{Z}$, with probability at least $1-\delta$ over the draw of $\mathcal{S}=\left\{z_{1},\dots,z_{m}\right\}\sim\mathcal{D}^{m}$, for all $f\in\mathcal{F}$, $\left|\mathbb{E}_{z\sim\mathcal{D}}[f(z)]-\frac{1}{m}\sum_{i=1}^{m}f\left(z_{i}\right)\right|\leq 2\widehat{\mathcal{R}}_{\mathcal{S}}(\mathcal{F})+4\kappa\sqrt{\frac{2}{m}\ln\frac{4}{\delta}}.$

To obtain data-dependent generalization guarantees, we rely on the following theorem, which follows from truncating the proof of Massart’s finite lemma (Massart, 2000), as observed by Riondato and Upfal (2015). For a function class $\mathcal{F}$ and a set $\mathcal{S}=\left\{z_{1},\dots,z_{m}\right\}\subseteq\mathcal{Z}$, we use the notation $\mathcal{F}(\mathcal{S})$ to denote the set of vectors $\left\{\left(f\left(z_{1}\right),\dots,f\left(z_{m}\right)\right):f\in\mathcal{F}\right\}$.

Theorem 4.2**.**

For a sample $\mathcal{S}$ of size $m$, suppose that $\mathcal{F}(\mathcal{S})$ is finite. Then

[TABLE]

Figure 7 shows a comparison of the worst-case versus data-dependent guarantees. The latter are significantly better. For ease of comparison, we compare the data-dependent bound provided by Theorem 4.2 with a variation of Theorem 3.7 in terms of Rademacher complexity. See Theorem D.1 in Appendix D for details.

5 Constraint satisfaction problems

In this section, we describe tree search for constraint satisfaction problems. The generalization guarantee from Section 3.3 also applies to tree search in this domain, as we describe in Appendix E.

A constraint satisfaction problem (CSP) is a tuple $(X,D,C)$, where $X=\left\{x_{1},\dots,x_{n}\right\}$ is a set of variables, $D=\left\{D_{1},\dots,D_{n}\right\}$ is a set of domains where $D_{i}$ is the set of values variable $x_{i}$ can take on, and $C$ is a set of constraints between variables. Each constraint in $C$ is a pair $\left(\left(x_{i_{1}},\dots,x_{i_{r}}\right),\psi\right)$ where $\psi$ is a function mapping $D_{i_{1}}\times\cdots\times D_{i_{r}}$ to $\{0,1\}$ for some $r\in[n]$ and some $i_{1},\dots,i_{r}\in[n]$. Given an assignment $\left(y_{1},\dots,y_{n}\right)\in D_{1}\times\cdots\times D_{n}$ of the variables in $X$, a constraint $\left(\left(x_{i_{1}},\dots,x_{i_{r}}\right),\psi\right)$ is satisfied if $\psi\left(y_{i_{1}},\dots,y_{i_{r}}\right)=1$. The goal is to find an assignment that maximizes the number of satisfied constraints.

The degree of a variable $x$, denoted $\deg(x)$, is the number of constraints involving $x$. The dynamic degree of (an unassigned variable) $x$ given a partial assignment $\bm{y}$, denoted $\textnormal{ddeg}(x,\bm{y})$ is the number of constraints involving $x$ and at least one other unassigned variable.

Example 5.1** (Graph $k$-coloring).**

Given a graph, the goal of this problem is to color its vertices using at most $k$ colors such that no two adjacent vertices share the same color. This problem can be formulated as a CSP, as illustrated by the following example. Suppose we want to 3-color the graph in Figure 8(a) using pink, green, and orange. The four vertices correspond to the four variables $X=\{x_{1},\dots,x_{4}\}$. The domain $D_{1}=\cdots=D_{4}=\{\text{pink, green, orange}\}$. The only constraints on this problem are that no two adjacent vertices share the same color. Therefore we define $\psi$ to be the “not equal” relation mapping $\{\text{pink, green, orange}\}\times\{\text{pink, green, orange}\}\to\{0,1\}$ such that $\psi(\omega_{1},\omega_{2})=\textbf{1}_{\{\omega_{1}\not=\omega_{2}\}}$. Finally, we define the set of constraints to be

[TABLE]

See Figure 8(b) for a coloring that satisfies all constraints ($y_{1}=y_{4}=\text{green, }y_{2}=\text{orange, and }y_{3}=\text{pink}).$

5.0.1 CSP tree search

CSP tree search begins by choosing a variable $x_{i}$ with domain $D_{i}$ and building $|D_{i}|$ branches, each one corresponding to one of the $|D_{i}|$ possible value assignments of $x$. Next, a node $Q$ of the tree is chosen, another variable $x_{j}$ is chosen, and $|D_{j}|$ branches from $Q$ are built, each corresponding to the possible assignments of $x_{j}$. The search continues and a branch is pruned if any of the constraints are not feasible given the partial assignment of the variables from the root to the leaf of that branch.

5.0.2 Variable selection in CSP tree search

As in MILP tree search, there are many variable selection policies researchers have suggested for choosing which variable to branch on at a given node. Typically, algorithms associate a score for branching on a given variable $x_{i}$ at node $Q$ in the tree $\mathcal{T}$, as in B&B. The algorithm then branches on the variable with the highest score. We provide several examples of common variable selection policies below.

deg/dom and ddeg/dom (Bessiere and Régin, 1996):

deg/dom corresponds to the scoring rule $\texttt{score}(\mathcal{T},Q,i)=\deg(x_{i})/|D_{i}|$ and ddeg/dom corresponds to the scoring rule $\texttt{score}(\mathcal{T},Q,i)=\textnormal{ddeg}(x_{i},\bm{y})/|D_{i}|$, where $\bm{y}$ is the assignment of variables from the root of $\mathcal{T}$ to $Q$.

Smallest domain (Haralick and Elliott, 1980):

In this case, $\texttt{score}(\mathcal{T},Q,i)=1/|D_{i}|$.

Our theory is for tree search and applies to both MILPs and CSPs. It applies both to lookahead approaches that require learning the weighting of the two children (the more promising and less promising child) and to approaches that require learning the weighting of several different scoring rules.

6 Conclusions and broader applicability

In this work, we studied machine learning for tree search algorithm configuration. We showed how to learn a nearly optimal mixture of branching (e.g., variable selection) rules. Through experiments, we showed that using the optimal parameter for one application domain when solving problem instances from a different application domain can lead to a substantial tree size blow up. We proved that this blowup can even be exponential. We provided the first sample complexity guarantees for tree search algorithm configuration. With only a small number of samples, the empirical cost incurred by using any mixture of two scoring rules will be close to its expected cost, where cost is an abstract measure such as tree size. We showed that using empirical Rademacher complexity, these bounds can be significantly tightened further. Through theory and experiments, we showed that learning to branch is practical and hugely beneficial.

While we presented the theory in the context of tree search, it also applies to other tree-growing applications. For example, it could be used for learning rules for selecting variables to branch on in order to construct small decision trees that correctly classify training examples. Similarly, it could be used for learning to branch in order to construct a desirable taxonomy of items represented as a tree—for example for representing customer segments in advertising day to day.

Acknowledgments.

This work was supported in part by the National Science Foundation under grants CCF-1422910, CCF-1535967, IIS-1618714, IIS-1718457, IIS-1617590, CCF-1733556, a Microsoft Research Faculty Fellowship, an Amazon Research Award, a NSF Graduate Research Fellowship, and the ARO under award W911NF-17-1-0082.

Appendix A Additional related work

As in this work, Khalil et al. [2016] study variable selection policies. Their goal is to find a variable selection strategy that mimics the behavior of the classic branching strategy known as strong branching while running faster than strong branching. They split a single MILP instance into a training and test set. The beginning of the B&B algorithm is the training phase: their algorithm assigns features to each node and observes the variable selected to branch on by strong branching using the product scoring rule (see the definition in Section 2.1.2). Collecting these features and variable observations for a pre-specified number of nodes, they thus amass a training set. Their algorithm then uses this training set together with a classic ranking algorithm from the machine learning literature [Joachims, 2002] to determine a function $f$ that, ideally, will imitate strong branching. In a bit more detail, on each node the algorithm encounters as it explores the rest of the tree, the algorithm assigns features $\bm{\phi}$ to that node, evaluates the function $f(\bm{\phi})$ which returns a variable to branch on, and the algorithm branches on that variable. The hope is that $f(\bm{\phi})$ matches the variable strong branching would choose to branch on at that node.

Other works have explored the use of machine learning techniques in the context of other aspects of B&B beyond variable selection. For example, He et al. [2014] use machine learning to speed up branch-and-bound, focusing on speeding up the node selection policy. Their work does not provide any learning-theoretic guarantees. Beyond node selection, He et al. [2014] also aim to learn a pruning policy, which determines whether to fathom a given node. This policy may occasionally fathom a node even when the branch contains the optimal solution, but ideally, this will happen infrequently or the resulting solution will be nearly optimal. Their learning algorithm takes as input a set of MIP instances and their optimal solutions. For each MIP in this set, the algorithm constructs a simple oracle which returns, given a set of B&B leaf nodes as input, the leaf whose feasible set contains the optimal solution. The learning algorithm uses the oracle to train the node selection and pruning policies using imitation learning. Ideally, the resulting node selection policy should choose the nodes the oracle selects, and the pruning policy should prune any node that does not contain the optimal solution in its feasible set.

Appendix B Variable selection policies

Appendix C Proofs from Section 3

Notation.

In the proofs in this section, we will use the following notation. Let $Q$ be a MILP instance. Suppose that we branch on $x_{i}$ and $x_{j}$, setting $x_{i}=0$ and $x_{j}=1$. We use the notation $Q_{i,j}^{-,+}$ to denote the resulting MILP. Similar, if we set $x_{i}=1$ and $x_{j}=0$, we denote the resulting MILP as $Q_{i,j}^{+,-}$.

See 3.1

Proof.

We populate the support of the distribution $\mathcal{D}$ by relying on two helpful theorems: Theorem 3.2 and C.4. In Theorem 3.2, we prove that for all $\mu^{*}\in\left(\frac{1}{3},\frac{2}{3}\right)$, there exists an infinite family $\mathcal{F}_{n,\mu^{*}}$ of MILP instances such that for any $Q\in\mathcal{F}_{n,\mu^{*}}$, if $\mu\in\left[0,\mu^{*}\right)$, then the scoring rule $\mu\texttt{score}_{1}+(1-\mu)\texttt{score}_{2}$ results in a B&B tree with $O(1)$ nodes and if $\mu\in\left(\mu^{*},1\right]$, the scoring rule results a tree with $2^{(n-4)/2}$ nodes. Conversely, in Theorem C.4, we prove that there exists an infinite family $\mathcal{G}_{n,\mu^{*}}$ of MILP instances such that for any $Q\in\mathcal{G}_{n,\mu^{*}}$, if $\mu\in\left[0,\mu^{*}\right)$, then the scoring rule $\mu\texttt{score}_{1}+(1-\mu)\texttt{score}_{2}$ results in a B&B tree with $2^{(n-5)/4}$ nodes and if $\mu\in\left(\mu^{*},1\right]$, the scoring rule results a tree with $O(1)$ nodes.

Now, let $Q_{a}$ be an arbitrary instance in $\mathcal{G}_{n,a}$ and let $Q_{b}$ be an arbitrary instance in $\mathcal{F}_{n,b}$. The theorem follows by letting $\mathcal{D}$ be a distribution such that $\Pr_{Q\sim\mathcal{D}}\left[Q=Q_{a}\right]=\Pr_{Q\sim\mathcal{D}}\left[Q=Q_{b}\right]=1/2.$ We know that if $\mu\in[0,1]\setminus(a,b)$, then the expected value of $\texttt{cost}\left(Q,\mu\texttt{score}_{1}+(1-\mu)\texttt{score}_{2}\right)$ is $\frac{1}{2}\left(O(1)+2^{(n-5)/4}\right)\geq 2^{(n-9)/4}.$ Meanwhile, if $\mu\in(a,b)$, then with probability 1,

[TABLE]

Throughout the proof of this theorem, we assume the node selection policy is depth-first search. We then prove that for any infeasible MILP, if NSP and NSP’ are two node selection policies and $\texttt{score}=\mu\texttt{score}_{1}+(1-\mu)\texttt{score}_{2}$ for any $\mu\in[0,1]$, then tree $\mathcal{T}$ B&B builds using NSP and score equals the tree $\mathcal{T}^{\prime}$ it builds using NSP’ and score (see Theorem C.9). Thus, the theorem holds for any node selection policy.∎

See 3.2

Proof.

For ease of notation in this proof, we will drop $\mathcal{T}$ from the input of the functions $\texttt{score}_{1}$ and $\texttt{score}_{2}$ since the scoring rules do not depend on $\mathcal{T}$, they only depend on the input MILP instance and variable.

For any constant $\gamma\geq 1$, let $\bm{c}_{1}=\gamma(1,2,\dots,n-3)$ and let $\bm{c}_{2}=\gamma\left(0,1.5,3-\frac{1}{2\mu^{*}}\right)$. Let $\bm{c}=\left(\bm{c}_{1},\bm{c}_{2}\right)\in\mathbb{R}^{n}$ be the concatenation of $\bm{c}_{1}$ and $\bm{c}_{2}$. Next, define the $n$-dimensional vectors $\bm{a}_{1}=2\sum_{i=1}^{n-3}\bm{e}_{i}$ and $\bm{a}_{2}=2\sum_{i=1}^{3}\bm{e}_{n-3+i}$, and let $A$ be a matrix whose first row is $\bm{a}_{1}$ and second row is $\bm{a}_{2}$. Let $Q_{\gamma,n}$ be the MILP

[TABLE]

We define $\mathcal{F}_{n,\mu^{*}}=\left\{Q_{n,\gamma}:\gamma\geq 1\right\}.$

Example C.1**.**

If $\gamma=1$ and $n=8$, then $Q_{\gamma,n}$ is

[TABLE]

For every even $n\geq 6$, both of the constraints $\bm{a}_{1}\cdot\bm{x}=2\sum_{i=1}^{n-3}x_{i}=n-3$ and $\bm{a}_{2}\cdot\bm{x}=2\sum_{i=1}^{3}x_{n-3+i}=3$ are infeasible for $\bm{x}\in\{0,1\}^{n}$ since $2\sum_{i=1}^{n-3}x_{i}$ and $2\sum_{i=1}^{3}x_{n-3+i}$ are even numbers but 3 and $n-3$ are odd numbers. The key idea of this proof is that if branch-and-bound branches on all variables in $\{x_{n-2},x_{n-1},x_{n}\}$ first, it will terminate upon making a tree of size at most $2^{3}=8$, since at most three branches are necessary to determine that $\bm{a}_{1}\cdot\bm{x}=2\sum_{i=1}^{3}x_{n-3+i}=3$ is infeasible. However, if branch-and-bound branches on all variables in $\{x_{1},\dots,x_{n-3}\}$ first, it will create a tree with exponential size before it terminates.

Lemma C.1**.**

Suppose $\mu<\mu^{*}$. Then for any MILP $Q_{\gamma,n}\in\mathcal{F}_{n,\mu^{*}}$, $\mu\texttt{score}_{1}+(1-\mu)\texttt{score}_{2}$ branches on all variables in $\{x_{n-2},x_{n-1},x_{n}\}$ before branching on variables in $\{x_{1},\dots,x_{n-3}\}$.

Proof of Lemma C.1.

For ease of notation, for the remainder of this proof, we drop the subscript $(\gamma,n)$ from $Q_{\gamma,n}$ and denote this MILP as $Q$. We first need to determine the form of $\breve{\bm{x}}_{Q}$, which is the optimal solution to the LP relaxation of $Q$. It is easiest to see how the LP relaxation will set the variables $x_{n-2}$, $x_{n-1}$, and $x_{n}$. The only constraints on these variables are that $2(x_{n-2}+x_{n-1}+x_{n})=3$ and that $x_{n-2},x_{n-1},x_{n}\in[0,1]$. Recall that $\bm{c}=\left(1,2,\dots,n-3,0,1.5,3-\frac{1}{2\mu^{*}}\right)$ is the vector defining the objective value of $Q$. Since $\mu^{*}>1/3$, we know that $1.5<3-\frac{1}{2\mu^{*}}$, which means that $c[n-2]<c[n-1]<c[n]$. Since the goal is to maximize $\bm{c}\cdot\bm{x}$, the LP relaxation will set $x_{n-2}=0$, $x_{n-1}=\frac{1}{2}$, and $x_{n}=1$. The logic for the first $n-3$ variables is similar. In this case, $c[1]<c[2]<\cdots<c[n-3]$, so the LP relaxation’s solution will put as much weight as possible on the variable $x_{n-3}$, then as much weight as possible on the variable $x_{n-4}$, and so on, putting as little weight as possible on the variable $x_{1}$ since it has the smallest corresponding objective coefficient $c[1]$. Since the only constraints on these variables are that $2\sum_{i=1}^{n-3}x_{i}=n-3$ and $x_{1},\dots,x_{n-3}\in[0,1]$, the LP objective value can set $\left\lfloor\frac{n-3}{2}\right\rfloor$ of the variables to 1, it can set one variable to $\frac{1}{2}$, and it has to set the rest of the variables to 0. Letting $i=\left\lceil\frac{n-3}{2}\right\rceil$, this means the LP relaxation will set the first $i-1$ variables $x_{1},\cdots x_{i-1}$ to zero, it will set $x_{i}=\frac{1}{2}$, and it will set $x_{i+1},\dots,x_{n-3}$ to 1. In other words,

[TABLE]

For example, if $n=8$, then $\breve{\bm{x}}_{Q}=\left(0,0,\frac{1}{2},1,1,0,\frac{1}{2},1\right)$. (See Figure 9(a).) Therefore, the only candidate variables to branch on are $x_{n-1}$ and $x_{i}$ where again, $i=\left\lceil(n-3)/2\right\rceil$.

To determine when variable B&B will branch on, we need to calculate $\breve{\bm{x}}_{Q_{i}^{-}}$ which is the solution to the LP relaxation of $Q$ with the additional constraint that $x_{i}=0$, as well as $x_{Q_{i}^{+}}$ which is the solution to the LP relaxation of $Q$ with the additional constraint that $x_{i}=1$, and $x_{Q_{n-1}^{-}}$ and $x_{Q_{n-1}^{+}}$. First, we will determine the form of the vectors $\breve{\bm{x}}_{Q_{n-1}^{-}}$ and $\breve{\bm{x}}_{Q_{n-1}^{+}}$. Suppose we set $x_{n-1}=0$. We now need to ensure that $2(x_{n-2}+0+x_{n})=3$ and that $x_{n-2},x_{n}\in[0,1]$. Since $c[n-2]=0<3-\frac{1}{2\mu^{*}}=c[n]$, the solution to the LP relaxation of $Q_{n-1}^{-}$ will set $x_{n-2}=\frac{1}{2}$ and $x_{n}=1$. (See Figure 9(b).) Similarly, if we set $x_{n-1}=1$, we now need to ensure that $2(x_{n-2}+1+x_{n})=3$ and that $x_{n-2},x_{n}\in[0,1]$. Therefore, the solution to the LP relaxation of $Q_{n-1}^{-}$ will set $x_{n-2}=0$ and $x_{n}=\frac{1}{2}$. (See Figure 9(c).) In other words, $\breve{\bm{x}}_{Q_{n-1}^{-}}=\breve{\bm{x}}_{Q}-\frac{1}{2}\bm{e}_{n-1}+\frac{1}{2}\bm{e}_{n-2}$ and $\breve{\bm{x}}_{Q_{n-1}^{+}}=\breve{\bm{x}}_{Q}+\frac{1}{2}\bm{e}_{n-1}-\frac{1}{2}\bm{e}_{n}$.

The argument for $\breve{\bm{x}}_{Q_{i}^{-}}$ and $\breve{\bm{x}}_{Q_{i}^{-}}$ is similar. Recall that in $\breve{\bm{x}}_{Q}$, the solution to the LP relaxation of the original MIP $Q$, we have that $\breve{x}_{Q}[i]=\frac{1}{2}$ since it is the median of the variables $x_{1},\dots,x_{n-3}$. For all $j>i$, we have that $\breve{x}_{Q}[j]=1$ and for all $j<i$, we have that $\breve{x}_{Q}[j]=0$. Suppose we set $x_{i}=0$. As before, the LP relaxation’s solution will put as much weight as possible on the variable $x_{n-3}$, then as much weight as possible on the variable $x_{n-4}$, and so on, putting as little weight as possible on the variable $x_{1}$ since it has the smallest corresponding objective coefficient $c[1]$. Since it cannot set $x_{i}=\frac{1}{2}$, it will set the next-best variable to $\frac{1}{2}$, which is $x_{i-1}$. (See Figure 9(d).) In other words, $\breve{\bm{x}}_{Q_{i}^{-}}=\breve{\bm{x}}_{Q}-\frac{1}{2}\bm{e}_{i}+\frac{1}{2}\bm{e}_{i-1}$. If we set $x_{i}=1$, the LP relaxation’s solution will have to take some weight away from the variables $x_{i+1},\dots,x_{n-3}$ since it needs to ensure that $2\sum_{i=1}^{n-3}x_{i}=n-3$. Therefore, it will set $x_{i+1}$ to $\frac{1}{2}$ and $x_{j}$ to 1 for all $j>i+1$. (See Figure 9(e).) In other words, $\breve{\bm{x}}_{Q_{i}^{+}}=\breve{\bm{x}}_{Q}+\frac{1}{2}\bm{e}_{i}-\frac{1}{2}\bm{e}_{i+1}$.

Therefore,

[TABLE]

This means that $\breve{c}_{Q}-\breve{c}_{Q_{i}^{-}}=\breve{c}_{Q}-\breve{c}_{Q_{i}^{+}}=\gamma/2$, $\breve{c}_{Q}-\breve{c}_{Q_{n-1}^{-}}=3\gamma/4$, and $\breve{c}_{Q}-\breve{c}_{Q_{n-1}^{+}}=\frac{\gamma}{4}\left(3-\frac{1}{\mu^{*}}\right)$. Therefore, $\mu\texttt{score}_{1}(Q,i)+(1-\mu)\texttt{score}_{2}(Q,i)=\gamma/2$ and $\mu\texttt{score}_{1}(Q,n-1)+(1-\mu)\texttt{score}_{2}(Q,n-1)=\frac{\mu\gamma}{4}\left(3-\frac{1}{\mu^{*}}\right)+\frac{3\gamma(1-\mu)}{4}=\frac{3\gamma}{4}-\frac{\mu\gamma}{4\mu^{*}}.$ This means that $\mu\texttt{score}_{1}(Q,i)+(1-\mu)\texttt{score}_{2}(Q,i)=\gamma/2<\frac{3\gamma}{4}-\frac{\mu\gamma}{4\mu^{*}}=\mu\texttt{score}_{1}(Q,n-1)+(1-\mu)\texttt{score}_{2}(Q,n-1)$ so long as $\mu<\mu^{*}$.

The next node branch-and-bound will explore is $Q_{n-1}^{-}$. The vector $\breve{\bm{x}}_{Q_{n-1}^{-}}$ has fractional values only in positions $i$ and $n-2$. Branching on $i$, we again have that $\breve{c}_{Q_{n-1}^{-}}-\breve{c}_{Q_{n-1,i}^{-,-}}=\breve{c}_{Q_{n-1}^{-}}-\breve{c}_{Q_{n-1,i}^{-,+}}=\gamma/2$. Branching on $n-2$, $Q_{n-1,n-2}^{-,-}$ is infeasible, so $\breve{c}_{Q_{n-1}^{-}}-\breve{c}_{Q_{n-1,n-2}^{-,-}}$ equals some large number $B\geq||\bm{c}||_{1}$. Next, $\breve{\bm{x}}_{Q_{n-1,n-2}^{-,+}}=\breve{\bm{x}}_{Q_{n-1}^{-}}+\frac{1}{2}\bm{e}_{n-2}-\frac{1}{2}\bm{e}_{n},$ so $\breve{c}_{Q_{n-1,n-2}^{-,+}}=\breve{c}_{Q_{n-1}^{-}}-\frac{\gamma}{2}\left(3-\frac{1}{2\mu^{*}}\right).$ Therefore, $\mu\texttt{score}_{1}\left(Q_{n-1}^{-},i\right)+(1-\mu)\texttt{score}_{2}\left(Q_{n-1}^{-},i\right)=\gamma/2$ and

[TABLE]

where the final inequality holds because $\mu^{*}<1<\frac{B-3\gamma/4}{B-3\gamma/2}$. Therefore, $x_{n-2}$ will be branched on next.

Since $Q_{n-1,n-2}^{-,-}$ is infeasible, the next node branch-and-bound will explore is $Q_{n-1,n-2}^{-,+}$. The vector $\breve{\bm{x}}_{Q_{n-1,n-2}^{-,+}}$ has fractional values only in positions $i$ and $n$. Both MILP instances $Q_{n-1,n-2,n}^{-,+,-}$ and $Q_{n-1,n-2,n}^{-,+,+}$ are infeasible, so $\mu\texttt{score}_{1}\left(Q_{n-1,n-2}^{-,+},n\right)+(1-\mu)\texttt{score}_{2}\left(Q_{n-1,n-2}^{-,+},n\right)=B$ whereas $\mu\texttt{score}_{1}\left(Q_{n-1,n-2}^{-,+},i\right)+(1-\mu)\texttt{score}_{2}\left(Q_{n-1,n-2}^{-,+},i\right)=\gamma/2$, as before. Therefore, branch-and-bound will branch on $x_{n}$ and fathom both children.

The next node branch-and-bound will explore is $Q_{n-1}^{+}$. The vector $\breve{\bm{x}}_{Q_{n-1}^{+}}$ has fractional values only in positions $i$ and $n$. Branching on $i$, we again have that $\breve{c}_{Q_{n-1}^{+}}-\breve{c}_{Q_{n-1,i}^{+,-}}=\breve{c}_{Q_{n-1}^{+}}-\breve{c}_{Q_{n-1,i}^{+,+}}=\gamma/2$. Branching on $x_{n}$, $\breve{\bm{x}}_{Q_{n-1,n}^{+,-}}=\breve{\bm{x}}_{Q_{n-1}^{+}}-\frac{1}{2}\bm{e}_{n}+\frac{1}{2}\bm{e}_{n-2},$ so $\breve{c}_{Q_{n-1,n}^{+,-}}=\breve{c}_{Q_{n-1}^{+}}-\frac{\gamma}{2}\left(3-\frac{1}{2\mu^{*}}\right).$ Meanwhile, $Q_{n-1,n}^{+,+}$ is infeasible, so $\breve{c}_{Q_{n-1}^{+}}-\breve{c}_{Q_{n-1,n-2}^{+,+}}=B$. Therefore, $\mu\texttt{score}_{1}\left(Q_{n-1}^{+},i\right)+(1-\mu)\texttt{score}_{2}\left(Q_{n-1}^{+},i\right)=\gamma/2$ and $\mu\texttt{score}_{1}\left(Q_{n-1}^{+},n\right)+(1-\mu)\texttt{score}_{2}\left(Q_{n-1}^{+},n\right)=\frac{\mu\gamma}{2}\left(3-\frac{1}{2\mu^{*}}\right)+(1-\mu)B>\gamma/2$. Therefore, $x_{n}$ will be branched on next.

The next node branch-and-bound will explore is $Q_{n-1,n}^{+,-}$. The vector $\breve{\bm{x}}_{Q_{n-1,n}^{+,-}}$ has fractional values only in positions $i$ and $n-2$. Both MILP instances $Q_{n-1,n,n-2}^{+,-,-}$ and $Q_{n-1,n,n-2}^{+,-,-}$ are infeasible, so $\mu\texttt{score}_{1}\left(Q_{n-1,n}^{-,+},n-2\right)+(1-\mu)\texttt{score}_{2}\left(Q_{n-1,n}^{-,+},n-2\right)=B$ whereas $\mu\texttt{score}_{1}\left(Q_{n-1,n-2}^{-,+},i\right)+(1-\mu)\texttt{score}_{2}\left(Q_{n-1,n-2}^{-,+},i\right)=\gamma/2$, as before. Therefore, branch-and-bound will branch on $x_{n-2}$ and fathom both children.

At this point, all children have been fathomed, so branch-and-bound will terminate. ∎

Lemma C.2**.**

Suppose $\mu>\mu^{*}$. Then for any MILP $Q_{\gamma,n}\in\mathcal{F}_{n,\mu^{*}}$, branch-and-bound with the scoring rule $\mu\texttt{score}_{1}+(1-\mu)\texttt{score}_{2}$ will create a tree of depth at least $2^{(n-5)/4}$.

Proof of Lemma C.2.

To prove this lemma, we use induction to show that on any path from the root of the branch-and-bound tree to a node of depth $i=\left\lfloor(n-3)/2\right\rfloor$, if $J$ are the set of indices branched on along that path, then $J\subseteq\{x_{1},\dots,x_{n-3}\}$. Even after branching on $i$ nodes from $\{x_{1},\dots,x_{n-3}\}$, the MILP will still be feasible, so the branch will not yet have been fathomed (since the original MILP is infeasible, a node will be fathomed only when it is infeasible). Therefore, branch-and-bound will continue down every branch to depth $\left\lfloor(n-3)/2\right\rfloor$, thus creating a tree with $2^{(n-4)/2}$ nodes.

Claim C.3**.**

On any path from the root of the branch-and-bound tree to a node of depth $i=\left\lfloor(n-3)/2\right\rfloor$, if $J$ are the set of indices branched on along that path, then $J\subseteq\{x_{1},\dots,x_{n-3}\}$.

Proof of Claim C.3.

We prove this claim by induction.

Inductive hypothesis. For $j\leq\left\lfloor(n-3)/2\right\rfloor$, let $J$ be the set of indices branched on along an arbitrary path of the branch-and-bound tree from the root to a node of depth $j$. Then $J\subseteq\{x_{1},\dots,x_{n-3}\}$.

Base case ($j=0$). As we saw in the proof of Lemma C.1, if $\mu>\mu^{*}$, then branch-and-bound will first branch on $x_{i}$ where $i=\left\lceil(n-3)/2\right\rceil$.

Inductive step. Let $j$ be an arbitrary index such that $0\leq j\leq i-1$. Let $J$ be the set of indices branched on along an arbitrary path of the branch-and-bound tree from the root to a node of depth $j$. We know from the inductive hypothesis that $J\subseteq\{x_{1},\dots,x_{n-3}\}$. Let $Q^{\prime}$ be the MILP at that node. Since $j\leq\left\lfloor(n-3)/2\right\rfloor-1$, we know that the LP relaxation of $Q^{\prime}$ is feasible. Let $z$ be the number of variables set to zero in $J$ and let $x_{p_{1}},x_{p_{2}},\dots,x_{p_{t}}$ be $\left\{x_{1},\dots,x_{n-3}\right\}\setminus J$ ordered such that $p_{k}<p_{k^{\prime}}$ for $k<k^{\prime}$. We know that the solution to the LP relaxation of $Q^{\prime}$ will have the first $i^{\prime}:=\left\lfloor(n-3)/2\right\rfloor-z$ variables $x_{p_{1}},\dots,x_{p_{i^{\prime}}}$ set to 0, it will set $x_{p_{i^{\prime}+1}}$ to 1/2, and it will set the remaining variables in $\left\{x_{1},\dots,x_{n-3}\right\}\setminus J$ to 1. Thus, the fractional variables are $x_{p_{i^{\prime}+1}}$ and $x_{n-1}$. Note that since $z\leq|J|\leq\left\lfloor(n-3)/2\right\rfloor-1$, $i^{\prime}=\left\lfloor(n-3)/2\right\rfloor-z\geq 1$.

Suppose we branch on $x_{p_{i^{\prime}+1}}$. If we set $x_{p_{i^{\prime}+1}}=0$, then the LP relaxation of $(Q^{\prime})_{p_{i^{\prime}+1}}^{-}$ will set $x_{p_{i^{\prime}}}$ to be 1/2 and otherwise the optimal solution will remain unchanged. Thus, $\breve{c}_{Q^{\prime}}-\breve{c}_{(Q^{\prime})_{p_{i^{\prime}+1}}^{-}}=\breve{c}_{Q^{\prime}}-\left(\breve{c}_{Q^{\prime}}-\gamma p_{i^{\prime}+1}/2+\gamma p_{i^{\prime}}/2\right)=\frac{\gamma\left(p_{i^{\prime}+1}-p_{i^{\prime}}\right)}{2}$. Meanwhile, if we set $x_{p_{i^{\prime}+1}}=1$, then the LP relaxation of $(Q^{\prime})_{p_{i^{\prime}+1}}^{-}$ will set $x_{p_{i^{\prime}+2}}$ to be 0 and otherwise the optimal solution will remain unchanged. Thus, $\breve{c}_{Q^{\prime}}-\breve{c}_{(Q^{\prime})_{p_{i^{\prime}+1}}^{+}}=\breve{c}_{Q^{\prime}}-\left(\breve{c}_{Q^{\prime}}+\gamma p_{i^{\prime}+1}/2-\gamma p_{i^{\prime}+2}/2\right)=\frac{\gamma\left(p_{i^{\prime}+2}-p_{i^{\prime}+1}\right)}{2}$. Suppose that $\breve{c}_{Q^{\prime}}-\breve{c}_{(Q^{\prime})_{p_{i^{\prime}+1}}^{+}}>\breve{c}_{Q^{\prime}}-\breve{c}_{(Q^{\prime})_{p_{i^{\prime}+1}}^{-}}$. Then

[TABLE]

Meanwhile, suppose that $\breve{c}_{Q^{\prime}}-\breve{c}_{(Q^{\prime})_{p_{i^{\prime}+1}}^{+}}\leq\breve{c}_{Q^{\prime}}-\breve{c}_{(Q^{\prime})_{p_{i^{\prime}+1}}^{-}}$. Then

[TABLE]

Meanwhile, as in the proof of Lemma C.1, $\mu\texttt{score}_{1}(Q^{\prime},n-1)+(1-\mu)\texttt{score}_{2}(Q^{\prime},n-1)=\frac{3\gamma}{4}-\frac{\mu\gamma}{4\mu^{*}}<\frac{\gamma}{2}$ so long as $\mu>\mu^{*}$. Thus, branch-and-bound will branch next on $x_{p_{i^{\prime}}}$. ∎

∎

∎

Theorem C.4**.**

Let

[TABLE]

For all even $n\geq 6$ and all $\mu^{*}\in\left(\frac{1}{3},\frac{2}{3}\right)$, there exists an infinite family $\mathcal{G}_{n,\mu^{*}}$ of MILP instances such that for any $Q\in\mathcal{G}_{n,\mu^{*}}$, if $\mu\in\left[0,\mu^{*}\right)$, then the scoring rule $\mu\texttt{score}_{1}+(1-\mu)\texttt{score}_{2}$ results in a branch-and-bound tree with $\Omega\left(2^{(n-5)/4}\right)$ nodes and if $\mu\in\left(\mu^{*},1\right]$, the scoring rule results a tree with $O(1)$ nodes.

Proof.

For any constant $\gamma\geq 1$, let $\bm{c}_{1}\in\mathbb{R}^{n-3}$ be a vector such that

[TABLE]

and let $\bm{c}_{2}=\left(1,2,3\right)$. Let $\bm{c}=\gamma\left(\bm{c}_{1},\bm{c}_{2}\right)\in\mathbb{R}^{n}$ be the concatenation of $\bm{c}_{1}$ and $\bm{c}_{2}$ multiplied with $\gamma$. For example, if $\gamma=1$ and $n=8$, then $\bm{c}=\left(0,0,1.5,3-\frac{1}{2\mu^{*}},3-\frac{1}{2\mu^{*}},1,2,3\right)$ Next, define the $n$-dimensional vectors $\bm{a}_{1}=2\sum_{i=1}^{n-3}\bm{e}_{i}$ and $\bm{a}_{2}=2\sum_{i=1}^{3}\bm{e}_{n-3+i}$, and let $A$ be a matrix whose first row is $\bm{a}_{1}$ and second row is $\bm{a}_{2}$. Let $Q_{\gamma,n}$ be the MILP

[TABLE]

We define $\mathcal{G}_{n,\mu^{*}}=\left\{Q_{n,\gamma}:\gamma\geq 1\right\}.$

For every even $n\geq 6$, $Q_{\gamma,n}$, both of the constraints $\bm{a}_{1}\cdot\bm{x}=2\sum_{i=1}^{n-3}x_{i}=n-3$ and $\bm{a}_{2}\cdot\bm{x}=2\sum_{i=1}^{3}x_{n-3+i}=3$ are infeasible for $\bm{x}\in\{0,1\}^{n}$ since $2\sum_{i=1}^{n-3}x_{i}$ and $2\sum_{i=1}^{3}x_{n-3+i}$ are even numbers but 3 and $n-3$ are odd numbers. The key idea of this proof is that if branch-and-bound branches on all variables in $\{x_{n-2},x_{n-1},x_{n}\}$ first, it will terminate upon making a tree of size at most $2^{3}=8$, since at most three branches are necessary to determine that $\bm{a}_{1}\cdot\bm{x}=2\sum_{i=1}^{3}x_{n-3+i}=3$ is infeasible. However, if branch-and-bound branches on all variables in $\{x_{1},\dots,x_{n-3}\}$ first, it will create a tree with exponential size before it terminates.

Lemma C.5**.**

Suppose $\mu>\mu^{*}$. Then for any MILP $Q_{\gamma,n}\in\mathcal{G}_{n,\mu^{*}}$, $\mu\texttt{score}_{1}+(1-\mu)\texttt{score}_{2}$ branches on all variables in $\{x_{n-2},x_{n-1},x_{n}\}$ before branching on variables in $\{x_{1},\dots,x_{n-3}\}$.

Proof of Lemma C.5.

For ease of notation, for the remainder of this proof, we drop the subscript $(\gamma,n)$ from $Q_{\gamma,n}$ and denote this MILP as $Q$. The optimal solution to the LP relaxation of $Q$ has the following form:

[TABLE]

For example, if $n=8$, then $\breve{\bm{x}}_{Q}=\left(0,0,\frac{1}{2},1,1,0,\frac{1}{2},1\right)$. Therefore, the only candidate variables to branch on are $x_{n-1}$ and $x_{i}$ where $i=\left\lceil(n-3)/2\right\rceil$. Branching on $x_{i}$, we have $\breve{\bm{x}}_{Q_{i}^{-}}=\breve{\bm{x}}_{Q}-\frac{1}{2}\bm{e}_{i}+\frac{1}{2}\bm{e}_{i-1}$ and $\breve{\bm{x}}_{Q_{i}^{+}}=\breve{\bm{x}}_{Q}+\frac{1}{2}\bm{e}_{i}-\frac{1}{2}\bm{e}_{i+1}$. Branching on $x_{n-1}$, we have $\breve{\bm{x}}_{Q_{n-1}^{-}}=\breve{\bm{x}}_{Q}-\frac{1}{2}\bm{e}_{n-1}+\frac{1}{2}\bm{e}_{n-2}$ and $\breve{\bm{x}}_{Q_{n-1}^{+}}=\breve{\bm{x}}_{Q}+\frac{1}{2}\bm{e}_{n-1}-\frac{1}{2}\bm{e}_{n}$. Therefore,

[TABLE]

This means that $\breve{c}_{Q}-\breve{c}_{Q_{i}^{-}}=3\gamma/4$, $\breve{c}_{Q}-\breve{c}_{Q_{i}^{+}}=\frac{\gamma}{4}\left(3-\frac{1}{\mu^{*}}\right)$, and $\breve{c}_{Q}-\breve{c}_{Q_{n-1}^{-}}=\breve{c}_{Q}-\breve{c}_{Q_{n-1}^{+}}=\gamma/2$. Therefore, $\mu\texttt{score}_{1}(Q,i)+(1-\mu)\texttt{score}_{2}(Q,i)=\frac{\mu\gamma}{4}\left(3-\frac{1}{\mu^{*}}\right)+\frac{3\gamma(1-\mu)}{4}=\frac{3\gamma}{4}-\frac{\mu\gamma}{4\mu^{*}}$ and $\mu\texttt{score}_{1}(Q,n-1)+(1-\mu)\texttt{score}_{2}(Q,n-1)=\gamma/2.$ This means that $\mu\texttt{score}_{1}(Q,i)+(1-\mu)\texttt{score}_{2}(Q,i)=\frac{3\gamma}{4}-\frac{\mu\gamma}{4\mu^{*}}<\gamma/2=\mu\texttt{score}_{1}(Q,n-1)+(1-\mu)\texttt{score}_{2}(Q,n-1)$ so long as $\mu>\mu^{*}$.

The next node branch-and-bound will explore is $Q_{n-1}^{-}$. The vector $\breve{\bm{x}}_{Q_{n-1}^{-}}$ has fractional values only in positions $i$ and $n-2$. Branching on $i$, we again have that $\breve{c}_{Q_{n-1}^{-}}-\breve{c}_{Q_{n-1,i}^{-,-}}=3\gamma/4$ and $\breve{c}_{Q_{n-1}^{-}}-\breve{c}_{Q_{n-1,i}^{-,+}}=\frac{\gamma}{4}\left(3-\frac{1}{\mu^{*}}\right)$. Branching on $n-2$, $Q_{n-1,n-2}^{-,-}$ is infeasible, so $\breve{c}_{Q_{n-1}^{-}}-\breve{c}_{Q_{n-1,n-2}^{-,-}}$ equals some large number $B\geq||\bm{c}||_{1}$. Next, $\breve{\bm{x}}_{Q_{n-1,n-2}^{-,+}}=\breve{\bm{x}}_{Q_{n-1}^{-}}+\frac{1}{2}\bm{e}_{n-2}-\frac{1}{2}\bm{e}_{n},$ so $\breve{c}_{Q_{n-1,n-2}^{-,+}}=\breve{c}_{Q_{n-1}^{-}}-\gamma.$ Therefore, $\mu\texttt{score}_{1}\left(Q_{n-1}^{-},i\right)+(1-\mu)\texttt{score}_{2}\left(Q_{n-1}^{-},i\right)=\frac{3\gamma}{4}-\frac{\mu\gamma}{4\mu^{*}}$ and

[TABLE]

where the final inequality holds because $\mu<1<\frac{B-3\gamma/4}{B-\left(\gamma+\gamma/\left(4\mu^{*}\right)\right)}$. Therefore, $x_{n-2}$ will be branched on next.

Since $Q_{n-1,n-2}^{-,-}$ is infeasible, the next node branch-and-bound will explore is $Q_{n-1,n-2}^{-,+}$. The vector $\breve{\bm{x}}_{Q_{n-1,n-2}^{-,+}}$ has fractional values only in positions $i$ and $n$. Since both MILP instances $Q_{n-1,n-2,n}^{-,+,-}$ and $Q_{n-1,n-2,n}^{-,+,+}$ are infeasible, so $\mu\texttt{score}_{1}\left(Q_{n-1,n-2}^{-,+},n\right)+(1-\mu)\texttt{score}_{2}\left(Q_{n-1,n-2}^{-,+},n\right)=B$ whereas $\mu\texttt{score}_{1}\left(Q_{n-1,n-2}^{-,+},i\right)+(1-\mu)\texttt{score}_{2}\left(Q_{n-1,n-2}^{-,+},i\right)=\frac{3\gamma}{4}-\frac{\mu\gamma}{4\mu^{*}}<B$. Therefore, branch-and-bound will branch on $x_{n}$ and fathom both children.

The next node branch-and-bound will explore is $Q_{n-1}^{+}$. The vector $\breve{\bm{x}}_{Q_{n-1}^{+}}$ has fractional values only in positions $i$ and $n$. Branching on $i$, we again have that $\breve{c}_{Q_{n-1}^{-}}-\breve{c}_{Q_{n-1,i}^{-,-}}=3\gamma/4$ and $\breve{c}_{Q_{n-1}^{-}}-\breve{c}_{Q_{n-1,i}^{-,+}}=\frac{\gamma}{4}\left(3-\frac{1}{\mu^{*}}\right)$. Branching on $x_{n}$, $\breve{\bm{x}}_{Q_{n-1,n}^{+,-}}=\breve{\bm{x}}_{Q_{n-1}^{+}}-\frac{1}{2}\bm{e}_{n}+\frac{1}{2}\bm{e}_{n-2},$ so $\breve{c}_{Q_{n-1,n}^{+,-}}=\breve{c}_{Q_{n-1}^{+}}-\gamma.$ Meanwhile, $Q_{n-1,n}^{+,+}$ is infeasible, so $\breve{c}_{Q_{n-1}^{+}}-\breve{c}_{Q_{n-1,n-2}^{+,+}}$ equals some large number $B\geq||\bm{c}||_{1}$. Therefore, $\mu\texttt{score}_{1}\left(Q_{n-1}^{+},i\right)+(1-\mu)\texttt{score}_{2}\left(Q_{n-1}^{+},i\right)=\frac{3\gamma}{4}-\frac{\mu\gamma}{4\mu^{*}}$ and $\mu\texttt{score}_{1}\left(Q_{n-1}^{+},n\right)+(1-\mu)\texttt{score}_{2}\left(Q_{n-1}^{+},n\right)=\mu\gamma+(1-\mu)B>\frac{3\gamma}{4}-\frac{\mu\gamma}{4\mu^{*}}$. Therefore, $x_{n}$ will be branched on next.

The next node branch-and-bound will explore is $Q_{n-1,n}^{+,-}$. The vector $\breve{\bm{x}}_{Q_{n-1,n}^{+,-}}$ has fractional values only in positions $i$ and $n-2$. Since both MILP instances $Q_{n-1,n,n-2}^{+,-,-}$ and $Q_{n-1,n,n-2}^{+,-,-}$ are infeasible, so $\mu\texttt{score}_{1}\left(Q_{n-1,n}^{-,+},n-2\right)+(1-\mu)\texttt{score}_{2}\left(Q_{n-1,n}^{-,+},n-2\right)=B$ whereas $\mu\texttt{score}_{1}\left(Q_{n-1,n-2}^{-,+},i\right)+(1-\mu)\texttt{score}_{2}\left(Q_{n-1,n-2}^{-,+},i\right)=\frac{3\gamma}{4}-\frac{\mu\gamma}{4\mu^{*}}<B$, as before. Therefore, branch-and-bound will branch on $x_{n-2}$ and fathom both children.

At this point, all children have been fathomed, so branch-and-bound will terminate. ∎

Lemma C.6**.**

Suppose $\mu<\mu^{*}$. Then for any MILP $Q_{\gamma,n}\in\mathcal{G}_{n,\mu^{*}}$, branch-and-bound with the scoring rule $\mu\texttt{score}_{1}+(1-\mu)\texttt{score}_{2}$ will create a tree of depth at least $2^{(n-5)/4}$.

Proof.

Let $i=\left\lceil(n-3)/2\right\rceil$. We first prove two useful claims.

Claim C.7**.**

Let $j$ be an even number such that $2\leq j\leq i-2$ and let $J=\left\{x_{i-j/2},\dots,x_{i+j/2-1}\right\}$. Suppose that B&B has branched on exactly the variables in $J$ and suppose that the number of variables set to 1 equals the number of variables set to 0. Then B&B will next branch on the variable $x_{i+j/2}$. Similarly, suppose $J=\left\{x_{i-j/2+1},x_{i-j/2+2},\dots,x_{i+j/2}\right\}$. Suppose that B&B has branched on exactly the variables in $J$ and suppose that the number of variables set to 1 equals the number of variables set to 0. Then B&B will next branch on the variable $x_{i-j/2}$.

Proof.

Let $Q$ be the MILP contained in the node at the end of the path. This proof has two cases.

Case 1: $J=\left\{x_{i-j/2},x_{i-j/2+1},\dots,x_{i+j/2-1}\right\}$.

In this case, there is a set $J_{<}=\left\{x_{1},\dots,x_{i-j/2-1}\right\}$ of $i-\frac{j}{2}-1$ variables smaller than $x_{i}$ that have not yet been branched on and there is a set $J_{>}=\left\{x_{i+j/2},\dots,x_{n-3}\right\}$ of $n-3-\left(i+\frac{j}{2}-1\right)=2i-1-\left(i+\frac{j}{2}-1\right)=i-\frac{j}{2}$ variables in $\{x_{i+1},\dots,x_{n-3}\}$ that have not yet been branched on. Since the number of variables set to 1 in $J$ equals the number of variables set to 0, the LP relaxation will set the $i-\frac{j}{2}-1$ variables in $J_{<}$ to 0, the $i-\frac{j}{2}-1$ variables in $J_{>}\setminus\left\{x_{i+j/2}\right\}$ to 1, and $x_{i+j/2}$ to $\frac{1}{2}.$ It will also set $x_{n-2}=0$, $x_{n-1}=\frac{1}{2}$, and $x_{n}=1$. Therefore, the two fractional variables are $x_{i+j/2}$ and $x_{n-1}$. Branching on $x_{i+j/2}$, we have $\breve{\bm{x}}_{Q_{i+j/2}^{-}}=\breve{\bm{x}}_{Q}-\frac{1}{2}\bm{e}_{i+j/2}+\frac{1}{2}\bm{e}_{i-j/2-1}$ and $\breve{\bm{x}}_{Q_{i+j/2}^{+}}=\breve{\bm{x}}_{Q}+\frac{1}{2}\bm{e}_{i+j/2}-\frac{1}{2}\bm{e}_{i+j/2+1}$. Branching on $x_{n-1}$, we have that $\breve{\bm{x}}_{Q_{n-1}^{-}}=\breve{\bm{x}}_{Q}-\frac{1}{2}\bm{e}_{n-1}+\frac{1}{2}\bm{e}_{n-2}$ and $\breve{\bm{x}}_{Q_{n-1}^{+}}=\breve{\bm{x}}_{Q}+\frac{1}{2}\bm{e}_{n-1}-\frac{1}{2}\bm{e}_{n}$. Therefore,

[TABLE]

This means that $\breve{c}_{Q}-\breve{c}_{Q_{i+j/2}^{-}}=\frac{\gamma}{2}\left(3-\frac{1}{2\mu^{*}}\right)$, $\breve{c}_{Q}-\breve{c}_{Q_{i+j/2}^{+}}=0$, and $\breve{c}_{Q}-\breve{c}_{Q_{n-1}^{-}}=\breve{c}_{Q}-\breve{c}_{Q_{n-1}^{+}}=\frac{\gamma}{2}$. Therefore, $\mu\texttt{score}_{1}(Q,i+j/2)+(1-\mu)\texttt{score}_{2}(Q,i+j/2)=\frac{\gamma\left(1-\mu\right)}{2}\left(3-\frac{1}{2\mu^{*}}\right)$ and $\mu\texttt{score}_{1}(Q,n-1)+(1-\mu)\texttt{score}_{2}(Q,n-1)=\frac{\gamma}{2}$. Since $\mu<\mu^{*}$ and $\mu^{*}\in\left(\frac{1}{3},\frac{1}{2}\right)$, we have that

[TABLE]

Therefore, $x_{i+j/2}$ will be branched on next.

Case 2: $J=\left\{x_{i-j/2+1},x_{i-j/2+2},\dots,x_{i+j/2}\right\}$.

In this case, there is a set $J_{<}=\left\{x_{1},\dots,x_{i-j/2}\right\}$ of $i-\frac{j}{2}$ variables smaller than $x_{i}$ that have not yet been branched on and a set $J_{>}=\left\{x_{i+j/2+1},\dots,x_{n-3}\right\}$ of $n-3-\left(i+\frac{j}{2}\right)=2i-1-\left(i+\frac{j}{2}\right)=i-\frac{j}{2}-1$ variables in $\{x_{i+1},\dots,x_{n-3}\}$ that have not yet been branched on. Since the number of variables set to 1 in $J$ equals the number of variables set to 0, the LP relaxation will set the $i-\frac{j}{2}-1$ variables in $J_{<}\setminus\left\{x_{i-j/2}\right\}$ to 0, the $i-\frac{j}{2}-1$ variables in $J_{>}\setminus\left\{x_{i+j/2}\right\}$ to 1, and $x_{i-j/2}$ to $\frac{1}{2}.$ It will also set $x_{n-2}=0$, $x_{n-1}=\frac{1}{2}$, and $x_{n}=1$. Therefore, the two fractional variables are $x_{i-j/2}$ and $x_{n-1}$. Branching on $x_{i-j/2}$, we have $\breve{\bm{x}}_{Q_{i-j/2}^{-}}=\breve{\bm{x}}_{Q}-\frac{1}{2}\bm{e}_{i-j/2}+\frac{1}{2}\bm{e}_{i-j/2-1}$ and $\breve{\bm{x}}_{Q_{i-j/2}^{+}}=\breve{\bm{x}}_{Q}+\frac{1}{2}\bm{e}_{i-j/2}-\frac{1}{2}\bm{e}_{i+j/2+1}$. Branching on $x_{n-1}$, we have that $\breve{\bm{x}}_{Q_{n-1}^{-}}=\breve{\bm{x}}_{Q}-\frac{1}{2}\bm{e}_{n-1}+\frac{1}{2}\bm{e}_{n-2}$ and $\breve{\bm{x}}_{Q_{n-1}^{+}}=\breve{\bm{x}}_{Q}+\frac{1}{2}\bm{e}_{n-1}-\frac{1}{2}\bm{e}_{n}$. Therefore,

[TABLE]

This means that $\breve{c}_{Q}-\breve{c}_{Q_{i-j/2}^{-}}=\frac{\gamma}{2}\left(3-\frac{1}{2\mu^{*}}\right)$, $\breve{c}_{Q}-\breve{c}_{Q_{i-j/2}^{+}}=0$, and $\breve{c}_{Q}-\breve{c}_{Q_{n-1}^{-}}=\breve{c}_{Q}-\breve{c}_{Q_{n-1}^{+}}=\frac{\gamma}{2}$ as in the previous case, so $x_{i-j/2}$ will be branched on next. ∎

Claim C.8**.**

Suppose that $J$ is the set of variables branched on along a path of depth $2\leq j\leq i-2$ where $j$ is odd and let $j^{\prime}=\lfloor j/2\rfloor$. Suppose that $J=\left\{x_{i-j^{\prime}},x_{i-j^{\prime}+1},\dots,x_{i+j^{\prime}}\right\}$. Moreover, suppose that the number of variables set to 1 in $J$ equals the number of variables set to 0, plus or minus 1. Then B&B will either branch on $x_{i-j^{\prime}-1}$ or $x_{i+j^{\prime}+1}$.

Proof.

Let $Q$ be the MILP contained a the end of the path. There are $i-j^{\prime}-1$ variables $J_{<}=\left\{x_{1},\dots,x_{i-j^{\prime}-1}\right\}$ that are smaller than $x_{i}$ that have not yet been branched on, and $n-3-(i+j^{\prime})=2i-1-(i+j^{\prime})=i-j^{\prime}-1$ variables $J_{>}=\left\{x_{i+j^{\prime}+1},\dots,x_{n-3}\right\}$ in $\left\{x_{i+1},\dots,x_{n-3}\right\}$ that have not yet been branched on. Let $z$ be the number of variables in $J$ set to 0 and let $o$ be the number of variables set to 1. This proof has two cases:

Case 1: $z=o+1$.

Since $z+o=j$, we know that $z=j^{\prime}+1$ and $o=j^{\prime}$. Therefore, the LP relaxation will set the variables in $J_{<}\setminus\left\{x_{i-j^{\prime}-1}\right\}$ to zero for a total of $\left|J_{<}\setminus\left\{x_{i-j^{\prime}-1}\right\}\right|+z=i-j^{\prime}-2+j^{\prime}+1=i-1$ zeros, it will set the variables in $J_{>}$ to one for a total of $\left|J_{>}\right|+o=i-j^{\prime}-1+j^{\prime}=i-1$ ones, and it will set $x_{i-j^{\prime}-1}$ to $\frac{1}{2}$. It will also set $x_{n-2}=0$, $x_{n-1}=\frac{1}{2}$, and $x_{n}=1$. Therefore, the two fractional variables are $x_{i-j^{\prime}-1}$ and $x_{n-1}$. Branching on $x_{i-j^{\prime}-1}$, we have $\breve{\bm{x}}_{Q_{i-j^{\prime}-1}^{-}}=\breve{\bm{x}}_{Q}-\frac{1}{2}\bm{e}_{i-j^{\prime}-1}+\frac{1}{2}\bm{e}_{i-j^{\prime}-2}$ and $\breve{\bm{x}}_{Q_{i-j^{\prime}-1}^{+}}=\breve{\bm{x}}_{Q}+\frac{1}{2}\bm{e}_{i-j^{\prime}-1}-\frac{1}{2}\bm{e}_{i+j^{\prime}+1}$. Branching on $x_{n-1}$, we have that $\breve{\bm{x}}_{Q_{n-1}^{-}}=\breve{\bm{x}}_{Q}-\frac{1}{2}\bm{e}_{n-1}+\frac{1}{2}\bm{e}_{n-2}$ and $\breve{\bm{x}}_{Q_{n-1}^{+}}=\breve{\bm{x}}_{Q}+\frac{1}{2}\bm{e}_{n-1}-\frac{1}{2}\bm{e}_{n}$. Therefore,

[TABLE]

This means that $\breve{c}_{Q}-\breve{c}_{Q_{i-j^{\prime}-1}^{-}}=0$, $\breve{c}_{Q}-\breve{c}_{Q_{i-j^{\prime}-1}^{+}}=\frac{\gamma}{2}\left(3-\frac{1}{2\mu^{*}}\right)$, and $\breve{c}_{Q}-\breve{c}_{Q_{n-1}^{-}}=\breve{c}_{Q}-\breve{c}_{Q_{n-1}^{+}}=\frac{\gamma}{2}$. Therefore, $\mu\texttt{score}_{1}(Q,i-j^{\prime}-1)+(1-\mu)\texttt{score}_{2}(Q,i-j^{\prime}-1)=\frac{\gamma(1-\mu)}{2}\left(3-\frac{1}{2\mu^{*}}\right)$ and $\mu\texttt{score}_{1}(Q,n-1)+(1-\mu)\texttt{score}_{2}(Q,n-1)=\frac{\gamma}{2}$. Since $\mu<\mu^{*}$ and $\mu^{*}\in\left(\frac{1}{3},\frac{1}{2}\right)$, we have that

[TABLE]

Therefore, $x_{i-j^{\prime}-1}$ will be branched on next.

Case 1: $z=o-1$.

Since $z+o=j$, we know that $z=j^{\prime}$ and $o=j^{\prime}+1$. Therefore, the LP relaxation will set the variables in $J_{<}$ to zero for a total of $\left|J_{<}\right|+z=i-j^{\prime}-1+j^{\prime}=i-1$ zeros, it will set the variables in $J_{>}\setminus\left\{x_{i+j^{\prime}+1}\right\}$ to one for a total of $\left|J_{>}\setminus\left\{x_{i+j^{\prime}+1}\right\}\right|+o=i-j^{\prime}-2+j^{\prime}+1=i-1$