TL;DR
This paper introduces a K-adaptability approach for two-stage mixed-integer robust optimization, enabling finite-dimensional approximation and efficient solution despite the challenges posed by discrete recourse decisions.
Contribution
It proposes a K-adaptability formulation combined with a branch-and-bound algorithm for solving complex two-stage robust optimization problems with mixed decisions.
Findings
Algorithm demonstrates asymptotic convergence.
Finite convergence achieved under certain conditions.
Performs well on benchmark datasets.
Abstract
We study two-stage robust optimization problems with mixed discrete-continuous decisions in both stages. Despite their broad range of applications, these problems pose two fundamental challenges: (i) they constitute infinite-dimensional problems that require a finite-dimensional approximation, and (ii) the presence of discrete recourse decisions typically prohibits duality-based solution schemes. We address the first challenge by studying a -adaptability formulation that selects candidate recourse policies before observing the realization of the uncertain parameters and that implements the best of these policies after the realization is known. We address the second challenge through a branch-and-bound scheme that enjoys asymptotic convergence in general and finite convergence under specific conditions. We illustrate the performance of our algorithm in numerical experimentsā¦
| Continuity | Convexity | Tractability | ||
| First-stage | problemĀ (1) | if feasible | if , convex | if , , convex and OBJ |
| problem | problemĀ (2) | if feasible | typically not | if , , convex and OBJ |
| Evaluation | problemĀ (1) | if OBJ | if convex and OBJ | if , convex and OBJ |
| problem | problemĀ (2) | if OBJ or CON | if convex and OBJ | if convex and OBJ |
| Second-stage | problemĀ (1) | if feasible | if convex | if convex |
| problem | problemĀ (2) | if feasible | always | always |
| Propositions A.1, B.1 | Propositions A.2, B.2 | Propositions A.3, B.3 | ||
| Opt (#) | Time (s) | Gap (%) | Opt (#) | Time (s) | Gap (%) | Opt (#) | Time (s) | Gap (%) | |
|---|---|---|---|---|---|---|---|---|---|
| 20 | 99 | 6 | 1.23 | 97 | 408 | 2.51 | 70 | 539 | 1.74 |
| 25 | 91 | 222 | 4.14 | 64 | 847 | 2.91 | 33 | 885 | 2.89 |
| 30 | 64 | 744 | 4.40 | 31 | 1,237 | 4.10 | 16 | 827 | 4.27 |
| 35 | 37 | 1,083 | 5.36 | 14 | 1,020 | 5.01 | 10 | 896 | 5.23 |
| 40 | 10 | 808 | 6.28 | 6 | 1,670 | 6.43 | 2 | 39 | 6.10 |
| 45 | 9 | 1,152 | 7.70 | 1 | 16 | 7.06 | 1 | 15 | 6.61 |
| 50 | 2 | 3,307 | 8.55 | 1 | 2,308 | 7.90 | 0 | ā | 7.10 |
| Opt (#) | Time (s) | Gap (%) | Opt (#) | Time (s) | Gap (%) | Opt (#) | Time (s) | Gap (%) | |
|---|---|---|---|---|---|---|---|---|---|
| 5 | 100 | 1 | ā | 100 | 1 | ā | 100 | 3 | ā |
| 10 | 100 | 1 | ā | 100 | 16 | ā | 100 | 149 | ā |
| 15 | 100 | 10 | ā | 99 | 566 | 0.33 | 69 | 2,245 | 1.42 |
| 20 | 100 | 419 | ā | 34 | 2,787 | 1.65 | 5 | 3,710 | 4.02 |
| 25 | 29 | 2,238 | 1.12 | 4 | 2,281 | 2.63 | 0 | ā | 6.22 |
| 30 | 1 | 188 | 3.01 | 1 | 6,687 | 3.35 | 0 | ā | 8.27 |
| Opt (#) | Time (s) | Gap (%) | Opt (#) | Time (s) | Gap (%) | Opt (#) | Time (s) | Gap (%) | |
|---|---|---|---|---|---|---|---|---|---|
| 5 | 100 | 1 | ā | 100 | 9 | ā | 98 | 80 | 3.14 |
| 10 | 100 | 3 | ā | 100 | 78 | ā | 98 | 938 | 1.92 |
| 15 | 100 | 62 | ā | 96 | 1,265 | 0.91 | 23 | 3,989 | 2.23 |
| 20 | 85 | 1,680 | 0.80 | 20 | 3,941 | 1.71 | 0 | ā | 4.94 |
| 25 | 12 | 3,363 | 2.29 | 1 | 2,693 | 3.34 | 0 | ā | 6.88 |
| 30 | 1 | 424 | 3.78 | 0 | ā | 4.73 | 0 | ā | 8.17 |
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
-Adaptability in Two-Stage Mixed-Integer Robust Optimization
Anirudh Subramanyam
Department of Chemical Engineering, Carnegie Mellon University, United States
Chrysanthos E.Ā Gounaris
Department of Chemical Engineering, Carnegie Mellon University, United States
Wolfram Wiesemann
Imperial College Business School, Imperial College London, United Kingdom
Abstract
We study two-stage robust optimization problems with mixed discrete-continuous decisions in both stages. Despite their broad range of applications, these problems pose two fundamental challenges: (i) they constitute infinite-dimensional problems that require a finite-dimensional approximation, and (ii) the presence of discrete recourse decisions typically prohibits duality-based solution schemes. We address the first challenge by studying a -adaptability formulation that selects candidate recourse policies before observing the realization of the uncertain parameters and that implements the best of these policies after the realization is known. We address the second challenge through a branch-and-bound scheme that enjoys asymptotic convergence in general and finite convergence under specific conditions. We illustrate the performance of our algorithm in numerical experiments involving benchmark data from several application domains.
Keywords: Robust Optimization, Two-Stage Problems, -Adaptability, Branch-and-Bound.
1 Introduction
Dynamic decision-making under uncertainty, where actions need to be taken both in anticipation of and in response to the realization of a priori uncertain problem parameters, arguably forms one of the most challenging domains of operations research and optimization theory. Despite intensive research efforts over the past six decades, many uncertainty-affected optimization problems resist solution, and even our understanding of the complexity of these problems remains incomplete.
In the last two decades, robust optimization has emerged as a promising methodology to counter some of the intricacies associated with decision-making under uncertainty. The rich theory on static robust optimization problems, in which all decisions have to be taken before the uncertainty is resolved, is summarized in [2, 4, 16]. However, dynamic robust optimization problems, in which some of the decisions can adapt to the observed uncertainties, are still poorly understood.
This paper is concerned with two-stage robust optimization problems of the form
[TABLE]
where , and constitute nonempty and bounded mixed-integer linear programming (MILP) representable sets, , and the functions , , and are affine. In problemĀ (1), the vector represents the first-stage (or āhere-and-nowā) decisions which are taken before the value of the uncertain parameter vector from within the uncertainty set is observed. The vector , on the other hand, denotes the second-stage (or āwait-and-seeā) decisions that can adapt to the realized value of . We emphasize that problemĀ (1) can have a random recourse, i.e., the recourse matrix may depend on the uncertain parameters . Moreover, we do not assume a relatively complete recourse; that is, for some first-stage decisions , there can be parameter realizations such that there is no feasible second-stage decision . Also, we do not assume that the sets , or are convex.
Remark 1** (Uncertain First-Stage Objective Coefficients).**
The assumption that is deterministic does not restrict generality. Indeed, problemĀ (1) accounts for uncertain first-stage objective coefficients if we augment the second-stage decisions to , replace the second-stage objective coefficients with and impose the constraint that .
Even in the special case where , and are linear programming (LP) representable, problemĀ (1) involves infinitely many decision variables and constraints, and it has been shown to be NP-hardĀ [24]. Nevertheless, problemĀ (1) simplifies considerably if the sets and are LP representable. For this setting, several approximate solution schemes have been proposed that replace the second-stage decisions with decision rules, i.e., parametric classes of linear or nonlinear functions of Ā [3, 15, 18, 19, 29]. If we further assume that , and are deterministic and is of simple form (e.g., a budget uncertainty set), a number of exact solution schemes based on Bendersā decomposition [10, 27, 34, 40] and semi-infinite programming [1, 39] have been developed.
ProblemĀ (1) becomes significantly more challenging if the set is not LP representable. For this setting, conservative MILP approximations have been developed in [21, 36] by partitioning the uncertainty set into hyperrectangles and restricting the continuous and integer recourse decisions to affine and constant functions of over each hyperrectangle, respectively. These a priori partitioning schemes have been extended to iterative partitioning approaches in [6, 32]. Iterative solution approaches based on decision rules have been proposed in [7, 8]. However, to the best of our knowledge, none of these approaches has been shown to converge to an optimal solution of problemĀ (1). For the special case where problemĀ (1) has a relatively complete recourse, , and are deterministic and the optimal value of the second-stage problem is quasi-convex over , the solution scheme of [39] has been extended in [41] to a nested semi-infinite approach that can solve instances of problemĀ (1) with MILP representable sets and to optimality in finite time.
Instead of solving problemĀ (1) directly, we study its -adaptability problem
[TABLE]
where and . ProblemĀ (2) determines non-adjustable second-stage policies here-and-now and subsequently selects the best of these policies in response to the observed value of . If all policies are infeasible for some realization , then the solution attains the objective value . By construction, the -adaptability problemĀ (2) bounds the two-stage robust optimization problemĀ (1) from above.
Our interest in problemĀ (2) is motivated by two observations. Firstly, problemĀ (2) has been shown to be a remarkably good approximation of problemĀ (1), both in theory and in numerical experimentsĀ [5, 25]. Secondly, and perhaps more importantly, the -adaptability problem conforms well with human decision-making, which tends to address uncertainty by developing a small number of contingency plans, rather than devising the optimal response for every possible future state of the world. For instance, practitioners may prefer a limited number of contingency plans to full flexibility in the second stage for operational (e.g., in production planning or logistics) or organizational (e.g., in emergency response planning) reasons.
The -adaptability problem was first studied inĀ [5], where the authors reformulate the -adaptability problem as a finite-dimensional bilinear program and solve it heuristically. The authors also show that the -adaptability problem is NP-hard even if , and are deterministic, and they develop necessary conditions for the -adaptability problemĀ (2) to outperform the static robust problem (where all decisions are taken here-and-now). The relationship between the -adaptability problemĀ (2) and static robust optimization is further explored inĀ [9] for the special case where and are deterministic. The authors show that the gaps between both problems and the two-stage robust optimization problemĀ (1) are intimately related to geometric properties of the uncertainty set . Finite-dimensional MILP reformulations for problemĀ (2) are developed inĀ [25] under the additional assumption that both the here-and-now decisions and the wait-and-see decisions are binary. The authors show that both the size of the reformulations as well as their gaps to the two-stage robust optimization problemĀ (1) depend on whether the uncertainty only affects the objective coefficients , or whether the constraint coefficients , and are uncertain as well. Finally, it is shown in [12, 13] that for polynomial time solvable deterministic combinatorial optimization problems, the associated instances of problemĀ (2) without first-stage decisions can also be solved in polynomial time if all of the following conditions hold: (i) is convex, (ii) only the objective coefficients are uncertain, and (iii) policies are sought. This result has been extended to discrete uncertainty sets inĀ [14], in which case pseudo-polynomial solution algorithms can be developed.
In this paper, we expand the literature on the -adaptability problem in two ways. From an analytical viewpoint, we compare the two-stage robust optimization problemĀ (1) with the -adaptability problemĀ (2) in terms of their continuity, convexity and tractability. We also investigate when the approximation offered by the -adaptability problem is tight, and under which conditions the two-stage robust optimization and -adaptability problems reduce to single-stage problems. From an algorithmic viewpoint, we develop a branch-and-bound scheme for the -adaptability problem that combines ideas from semi-infinite and disjunctive programming. We establish conditions for its asymptotic and finite time convergence; we show how it can be refined and integrated into state-of-the-art MILP solvers; and, we present a heuristic variant that can address large-scale instances. In contrast to existing approaches, our algorithm can handle mixed continuous and discrete decisions in both stages as well as discrete uncertainty, and allows for modeling continuous second-stage decisions via a novel class of highly flexible piecewise affine decision rules. Extensive numerical experiments on benchmark data from various application domains indicate that our algorithm is highly competitive with state-of-the-art solution schemes for problemsĀ (1) andĀ (2).
The remainder of this paper develops as follows. SectionĀ 2 analyzes the geometry and tractability of problemsĀ (1) andĀ (2), and it proposes a novel class of piecewise affine decision rules that can be modeled as an instance of problemĀ (2) with continuous second-stage decisions. SectionĀ 3 develops a branch-and-bound algorithm for the -adaptability problem and analyzes its convergence. We present numerical results in SectionĀ 4, and we offer concluding remarks in SectionĀ 5.
Notation. Vectors and matrices are printed in bold lowercase and uppercase letters, respectively, while scalars are printed in regular font. We use to denote the unit basis vector and to denote the vector whose components are all ones, respectively; their dimensions will be clear from the context. The row vector of a matrix is denoted by . For a logical expression , we define as the indicator function which takes a value of is is true and [math] otherwise.
2 Problem Analysis
In this section, we analyze the geometry and tractability of the two-stage robust optimization problemĀ (1) and its associated -adaptability problemĀ (2). Specifically, we characterize the continuity, convexity and tractability of both problems, as well as their relationship to the static robust optimization problem where all decisions are taken here-and-now. We also show how the -adaptability problem with continuous second-stage decisions enables us to approximate the two-stage robust optimization problemĀ (1) through highly flexible piecewise affine decision rules.
TableĀ 1 summarizes our theoretical results. For the sake of brevity, we present the formal statements of these results and their proofs as supplementary material at the end of the paper. In the table, the first-stage problem refers to the overall problemsĀ (1) andĀ (2), the evaluation problem refers to the maximization over for a fixed first-stage decision, and the second-stage problem refers to the inner minimization over or for a fixed first-stage decision and a fixed realization of the uncertain problem parameters. The table reveals that despite significant differences in their formulations, the problemsĀ (1) andĀ (2) behave very similarly. The most significant difference is caused by the replacement of the optimization over the second-stage decisions in problemĀ (1) with the selection of a candidate policy in problemĀ (2). This ensures that the second-stage problem inĀ (2) is always continuous, convex and tractable, whereas the first-stage problem inĀ (2) fails to be convex even if and are convex. Moreover, in contrast to problemĀ (1), the evaluation problem inĀ (2) remains continuous as long as either the objective function or the constraints are unaffected by uncertainty. For general problem instances, however, neither of the two evaluation problems is continuous. As we will see in SectionĀ 3.2, this directly impacts the convergence of our branch-and-bound algorithm, which only takes place asymptotically in general. Note that the convexity of the problemsĀ (1) andĀ (2) does not depend on the shape of the uncertainty set .
2.1 Incorporating Decision Rules in the -Adaptability Problem
Although the -adaptability problemĀ (2) selects the best candidate policy in response to the observed parameter realization , the policies āonce selected in the first stageāno longer depend on . This lack of dependence on the uncertain problem parameters can lead to overly conservative approximations of the two-stage robust optimization problemĀ (1) when the second-stage decisions are continuous. In this section, we show how the -adaptability problemĀ (2) can be used to generalize affine decision rules, which are commonly used to approximate continuous instances of the two-stage robust optimization problemĀ (1). We note that existing schemes, such asĀ [13, 25], cannot be used for this purpose as they require the wait-and-see decisions to be binary.
Throughout this section, we assume that problemĀ (1) has purely continuous second-stage decisions (that is, is LP representable), a deterministic objective function (that is, for all ) and fixed recourse (that is, for all ). The assumption of continuous second-stage decisions allows us to assume, without loss of generality, that as any potential restrictions can be absorbed in the second-stage constraints.
The affine decision rule approximation to the two-stage robust optimization problemĀ (1) is
[TABLE]
where indicates that for some and , seeĀ [3]. This problem provides a conservative approximation to the two-stage robust optimization problemĀ (1) since we replace the space of all (possibly non-convex and discontinuous) second-stage policies with the subspace of all affine second-stage policies . In a similar spirit, we define the subspace of all piecewise affine decision rules with pieces as
[TABLE]
Note that our earlier definition of is identical to our definition of if . For , the decision rules may be non-convex and discontinuous, and the regions where is affine may be non-closed and non-convex. We highlight that the points of nonlinearity are determined by the optimization problem. This is in contrast to many existing solution schemes for piecewise affine decision rules, such as [8, 15, 18, 19], where these points are specified ad hoc by the decision-maker.
Observation 1**.**
The piecewise affine decision rule problem with fixed recourse
[TABLE]
is equivalent to the -adaptability problem with random recourse
[TABLE]
where \hat{\mathcal{Y}}=\Big{\{}(\bm{y}^{0},\bm{Y},\bm{z})\in\mathbb{R}^{N_{2}}\times\mathbb{R}^{N_{2}\times N_{p}}\times(\mathbb{R}^{N_{p}}\times\mathbb{R}^{N_{p}L})\,:\,\bm{z}=(\bm{z}_{1},\bm{z}_{2})\text{ with }\bm{z}_{1}=\bm{Y}^{\top}\bm{d}\text{ and }\bm{z}_{2}=[\bm{w}_{1}^{\top}\bm{Y}\;\ldots\;\bm{w}_{L}^{\top}\bm{Y}]^{\top}\Big{\}}, and \bm{\hat{W}}(\bm{\xi})=\text{\emph{diag}}\big{(}\bm{\xi}^{\top},\,\ldots,\,\bm{\xi}^{\top}\big{)}\in\mathbb{R}^{L\times N_{p}L}.
Proof.
ProblemĀ (4) is infeasible if and only if for every and there is a such that for all , which in turn is the case if and only if for every and , there is a such that for all ; that is, if and only if problemĀ (3) is infeasible. We thus assume that bothĀ (3) andĀ (4) are feasible. In this case, we verify that every feasible solution to problemĀ (4) gives rise to a feasible solution , where and is any element of \mathop{\arg\min}\limits_{k\in\mathcal{K}}\Big{\{}\bm{c}^{\top}\bm{x}+\bm{d}^{\top}\bm{y}_{k}^{0}+\bm{\hat{d}}(\bm{\xi})^{\top}\bm{z}_{k1}\,:\,\bm{T}(\bm{\xi})\bm{x}+\bm{W}\bm{y}_{k}^{0}+\bm{\hat{W}}(\bm{\xi})\bm{z}_{k2}\leq\bm{h}(\bm{\xi})\Big{\}}, in problemĀ (3) that attains the same worst-case objective value. Similarly, every optimal solution to problemĀ (3) gives rise to an optimal solution , where with and , , in problemĀ (4). Hence,Ā (3) andĀ (4) share the same optimal value and the same sets of optimal solutions. ā
We close with an example that illustrates the benefits of piecewise affine decision rules.
Example 1**.**
Consider the following instance of the two-stage robust optimization problemĀ (1), which has been proposed inĀ [20, SectionĀ 5.2]:
[TABLE]
The optimal second-stage policy is , where , and it results in the optimal second-stage value function with a worst-case objective value of , see FigureĀ 1. The best affine decision rule is , and it results in the constant second-stage value function . The best -adaptable affine decision rule , on the other hand, is given by
[TABLE]
and it results in the constant second-stage value function . Thus, -adaptable affine decision rules are optimal in this example. FigureĀ 2 illustrates the optimal value, the affine approximation and the -adaptable affine approximation of the decision variable .
The piecewise affine decision rules presented here can be readily combined with discrete second-stage decisions. For the sake of brevity, we omit the details of this straightforward extension.
3 Solution Scheme
Our solution scheme for the -adaptability problemĀ (2) is based on a reformulation as a semi-infinite disjunctive program which we present next.
Observation 2**.**
The -adaptability problemĀ (2) is equivalent to
[TABLE]
Moreover, if some of the constraints in problemĀ (5) are deterministic, i.e., they do not depend on , then they can be moved outside the disjunction and instead be enforced for all .
In the following, we stipulate that the optimal value ofĀ (5) is whenever it is infeasible.
Proof of ObservationĀ 2.
ProblemĀ (2) is infeasible if and only if (iff) for every and there is a such that for all , which in turn is the case iff for every and , the disjunction inĀ (5) is violated for at least one ; that is, iff problemĀ (5) is infeasible. We thus assume that bothĀ (2) andĀ (5) are feasible. In this case, one readily verifies that every feasible solution to problemĀ (2) gives rise to a feasible solution , where \theta=\sup\limits_{\bm{\xi}\in\Xi}\;\inf\limits_{k\in\mathcal{K}}\big{\{}\bm{c}^{\top}\bm{x}+\bm{d}(\bm{\xi})^{\top}\bm{y}_{k}:\bm{T}(\bm{\xi})\bm{x}+\bm{W}(\bm{\xi})\bm{y}_{k}\leq\bm{h}(\bm{\xi})\big{\}}, in problemĀ (5) with the same objective value. Likewise, any optimal solution to problemĀ (5) corresponds to an optimal solution in problemĀ (2). Hence,Ā (2) andĀ (5) share the same optimal value and the same sets of optimal solutions.
We now claim that if , and for all , where , then problemĀ (5) is equivalent to
[TABLE]
By construction, every feasible solution to problemĀ () is feasible in problemĀ (5). Conversely, fix any feasible solution to problemĀ (5) and assume that for some and . In that case, the disjunct inĀ (5) is violated for every realization . We can therefore replace with a different candidate policy that satisfies for all without sacrificing feasibility. (Note that such a candidate policy exists since is assumed to be feasible inĀ (5).) Replacing any infeasible policy in this way results in a solution that is feasible in problemĀ (). ā
ProblemĀ (5) cannot be solved directly as it contains infinitely many disjunctive constraints. Instead, our solution scheme iteratively solves a sequence of (increasingly tighter) relaxations of this problem that are obtained by enforcing the disjunctive constraints over finite subsets of . Whenever the solution of such a relaxation violates the disjunction for some realization , we create subproblems that enforce the disjunction associated with to be satisfied by the disjunct, . Our solution scheme is reminiscent of discretization methods employed in semi-infinite programming, which iteratively replace an infinite set of constraints with finite subsets and solve the resulting discretized problems. Indeed, our scheme can be interpreted as a generalization of Kelleyās cutting-plane method [11, 28] applied to semi-infinite disjunctive programs. In the special case where , our method reduces to the cutting-plane method for (static) robust optimization problems proposed in [31].
In the remainder of this section, we describe our basic branch-and-bound scheme (SectionĀ 3.1), we study its convergence (SectionĀ 3.2), we discuss algorithmic variants to the basic scheme that can enhance its numerical performance (SectionĀ 3.3), and we present a heuristic variant that can address problems of larger scale (SectionĀ 3.4).
3.1 Branch-and-Bound Algorithm
Our solution scheme iteratively solves a sequence of scenario-based -adaptability problems and separation problems. We define both problems first, and then we describe the overall algorithm.
The Scenario-Based -Adaptability Problem.
For a collection of finite subsets of the uncertainty set , we define the scenario-based -adaptability problem as
[TABLE]
If and are convex, problemĀ (6) is an LP; otherwise, it is an MILP. The problem is closely related to a relaxation of the semi-infinite disjunctive programĀ (5) that enforces the disjunction only over the realizations . More precisely, problemĀ (6) can be interpreted as a restriction of that relaxation which requires the candidate policy to be worst-case optimal for all realizations , . We obtain an optimal solution to the relaxed semi-infinite disjunctive program by solving for all partitions of and reporting the optimal solution of the problem with the smallest objective value.
If for all , then problemĀ (6) is unbounded, and we stipulate that its optimal value is and that its optimal value is attained by any solution with . Otherwise, if problemĀ (6) is infeasible for , then we define its optimal value to be . In all other cases, the optimal value of problemĀ (6) is finite and it is attained by an optimal solution since and are compact.
Remark 2** (Decomposability).**
For -adaptability problems without first-stage decisions , problemĀ (6) decomposes into scenario-based static robust optimization problems that are only coupled through the constraints referencing the epigraph variable . In this case, we can recover an optimal solution to problemĀ (6) by solving each of the static problems individually and identifying the optimal as the maximum of their optimal values.
The Separation Problem.
For a feasible solution to the scenario-based -adaptability problemĀ (6), we define the separation problem as
[TABLE]
for and . Whenever it is positive, the innermost maximum in the definition of records the maximum constraint violation of the candidate policy under the parameter realization . Likewise, the quantity denotes the excess of the objective value of under the realization over the current candidate value of the worst-case objective, . Thus, is strictly positive if and only if every candidate policy either is infeasible or results in an objective value greater than under the realization . Whenever is finite, the separation problem is feasible and bounded, and it has an optimal solution since is nonempty and compact. Otherwise, we have , and the optimal value is attained by any .
Observation 3**.**
The separation problemĀ (7) is equivalent to the MILP
[TABLE]
This problem can be solved in polynomial time if is convex and the number of policies is fixed.
Proof.
Fix any feasible solution to the scenario-based -adaptability problemĀ (6). For every , we can construct a feasible solution to problemĀ (8) with by setting if for all and for otherwise (where ties can be broken arbitrarily). We thus conclude that is less than or equal to the optimal value of problemĀ (8). Likewise, every feasible solution to problemĀ (8) satisfies \zeta\leq\max\big{\{}\bm{c}^{\top}\bm{x}+\bm{d}(\bm{\xi})^{\top}\bm{y}_{k}-\theta,\;\max\limits_{l\in\{1,\ldots,L\}}\;\{\bm{t}_{l}(\bm{\xi})^{\top}\bm{x}+\bm{w}_{l}(\bm{\xi})^{\top}\bm{y}_{k}-h_{l}(\bm{\xi})\}\big{\}} for all ; that is, . Thus, the optimal value of problemĀ (8) is less than or equal to as well.
If the number of policies is fixed and the uncertainty set is convex, then problemĀ (8) can be solved by enumerating all possible choices for , solving the resulting linear programs in and and reporting the solution with the maximum value of . ā
The Algorithm.
Our solution scheme solves a sequence of scenario-based -adaptability problemsĀ (6) over monotonically increasing scenario sets , . At each iteration, the separation problemĀ (8) identifies a new scenario to be added to these sets.
Initialize. Set (node set), where with for all (root node). Set (incumbent solution). 2. 2.
Check convergence. If , then stop and declare infeasibility (if ) or report as an optimal solution to problemĀ (2). 3. 3.
Select node. Select a node from . Set . 4. 4.
Process node. Let be an optimal solution to the scenario-based -adaptability problemĀ (6). If , then go to StepĀ 2. 5. 5.
Check feasibility. Let be an optimal solution to the separation problemĀ (8). If , then set and go to StepĀ 2. 6. 6.
Branch. Instantiate new nodes as follows: for each . Set and go to StepĀ 3.
Our branch-and-bound algorithm can be interpreted as an uncertainty set partitioning scheme. For a solution in StepĀ 4, the sets
[TABLE]
describe the regions of the uncertainty set for which at least one of the candidate policies is feasible and results in an objective value smaller than or equal to . StepĀ 5 of the algorithm attempts to identify a realization for which every candidate policy either is infeasible or results in an objective value that exceeds . If there is no such realization, then the solution is feasible in the -adaptability problemĀ (2). Otherwise, StepĀ 6 assigns the realization to each scenario subset , , in turn. FigureĀ 3 illustrates our solution scheme.
3.2 Convergence Analysis
We now establish the correctness of our branch-and-bound scheme, as well as conditions for its asymptotic and finite convergence.
Theorem 1** (Correctness).**
If the branch-and-bound scheme terminates, then it either returns an optimal solution to problemĀ (2) or correctly identifies the latter as infeasible.
Proof.
We first show that if the problem instance is infeasible, then the algorithm terminates with the incumbent solution . Indeed, the algorithm can only update the incumbent solution in StepĀ 5 if the objective value of the separation problem is non-positive. By construction, this is only possible if the algorithm has determined a feasible solution.
We now show that for feasible problem instances, the algorithm terminates with an optimal solution of problemĀ (2). To this end, assume that is an optimal solution of problemĀ (2) with objective value . Let be the set of all nodes of the branch-and-bound tree for which is feasible in the corresponding scenario-based -adaptability problemĀ (6). Note that since is feasible in the root node. Let be the set of those nodes which have children in and consider the set ; by construction, we have . Consider an arbitrary node . By definition of , our algorithm has not branched . Since has been selected in StepĀ 3, this is only possible if either (i) has been fathomed in StepĀ 4 or if (ii) has been fathomed in StepĀ 5. In the former case, the solution must have been weakly dominated by the incumbent solution , which therefore must be optimal as well. In the latter case, the incumbent solution must have been updated to . ā
We now show that our branch-and-bound scheme converges asymptotically to an optimal solution of the -adaptability problemĀ (2). Our result has two implications: (i) for infeasible problem instances, the algorithm always terminates after finitely many iterations, i.e., infeasibility is detected in finite time; (ii) for feasible problem instances, the algorithm eventually only inspects solutions in the neighborhood of optimal solutions.
Theorem 2** (Asymptotic Convergence).**
Every accumulation point of the solutions to the scenario-based -adaptability problemĀ (6) in an infinite branch of the branch-and-bound tree gives rise to an optimal solution of the -adaptability problemĀ (2) with objective value .
Proof.
We denote by and the sequences of optimal solutions to the scenario-based -adaptability problem in StepĀ 4 and the separation problem in StepĀ 5 of the algorithm, respectively, that correspond to the node sequence , , of some infinite branch of the branch-and-bound tree. Since , and are compact, the Bolzano-Weierstrass theorem implies that and each have at least one accumulation point.
We first show that every accumulation point of the sequence corresponds to a feasible solution of the -adaptability problemĀ (2) with objective value . By possibly going over to subsequences, we can without loss of generality assume that the two sequences and converge themselves to and , respectively. Assume now that does not correspond to a feasible solution of the -adaptability problemĀ (2) with objective value . Then there is such that for some . By construction of the separation problemĀ (8), this implies that
[TABLE]
for all sufficiently large. By taking limits and exploiting the continuity of , we conclude that
[TABLE]
Note, however, that since for some . Since the sequence also converges to and converges to , we thus conclude that , which yields the desired contradiction.
We now show that every accumulation point of the sequence corresponds to an optimal solution of the -adaptability problemĀ (2) with objective value . Assume to the contrary that is feasible but suboptimal. Then there is a feasible solution with that either (i) is used to update the incumbent solution after finitely many iterations, or (ii) constitutes the accumulation point of another infinite sequence . In the first case, the objective values of the scenario-based -adaptability problems will be arbitrarily close to for sufficiently large, which implies that the corresponding nodes will be fathomed in StepĀ 4. Similarly, in the second case the objective values and of the scenario-based -adaptability problems will be arbitrarily close to and , respectively, for sufficiently large. Since , the algorithm will fathom the tree nodes corresponding to the sequence in StepĀ 4. The result now follows since both cases contradict the assumption that is an accumulation point. ā
TheoremĀ 2 guarantees that after sufficiently many iterations of the algorithm, our scheme generates feasible solutions that are close to an optimal solution of the -adaptability problemĀ (2). In general, our algorithm may not converge after finitely many iterations. In the following, we discuss a class of problem instances for which finite convergence is guaranteed.
Theorem 3** (Finite Convergence).**
The branch-and-bound scheme terminates after finitely many iterations, if has finite cardinality and only the objective function in problemĀ (2) is uncertain.
Proof.
If only the objective function in the -adaptability problemĀ (2) is uncertain, then the corresponding semi-infinite disjunctive programĀ (5) can be written as
[TABLE]
see ObservationĀ 2. Thus, the scenario-based -adaptability problemĀ (6) becomes
[TABLE]
and the separation problemĀ (7) can be written as
[TABLE]
We now show that if has finite cardinality, then our branch-and-bound algorithm terminates after finitely many iterations. To this end, assume that this is not the case, and let , be some rooted branch of the tree with infinite length. We denote by and the corresponding sequences of optimal solutions to the master and the separation problem, respectively. Since has finite cardinality, we must have for some .
The solution satisfies since , , is a branch of infinite length. Since , we thus conclude that
[TABLE]
Since is optimal in the separation problem and , we have
[TABLE]
However, since the node is a descendant of the node , we must have for some . This, along with the fact that is a feasible solution to the master problem and that , implies that
[TABLE]
This yields the desired contradiction and proves the theorem. ā
We note that the assumption of deterministic constraints is critical in the previous statement.
Example 2**.**
Consider the following instance of the -adaptability problemĀ (2):
[TABLE]
On this instance, our branch-and-bound algorithm generates a tree in which all branches have finite length, except (up to permutations) the sequence of nodes , where and (\Xi_{1}^{\ell},\Xi_{2}^{\ell})=\left(\big{\{}\xi^{0}2^{-i}\,:\,i=0,1,\ldots,\ell-1\big{\}},\{0\}\right), , for some . For the node , , the optimal solution of the scenario-based -adaptability problemĀ (6) is , while the optimal solution of the separation problem is . Thus, our branch-and-bound algorithm does not terminate after finitely many iterations.
We note that every practical implementation of our branch-and-bound scheme will fathom nodes in StepĀ 5 whenever the objective value of the separation problemĀ (7) is sufficiently close to zero (within some -tolerance). This ensures that the algorithm terminates in finite time in practice. Indeed, in ExampleĀ 2 the objective value of the separation problem is less than for all nodes with , and our branch-and-bound algorithm will fathom the corresponding path of the tree after iterations if we seek -precision solutions.
3.3 Improvements to the Basic Algorithm
The algorithm of SectionĀ 3.1 serves as a blueprint that can be extended in multiple ways. In the following, we discuss three enhancements that improve the numerical performance of our algorithm.
Breaking Symmetry.
For any feasible solution of the -adaptability problemĀ (2), every solution , where is one of the permutations of the second-stage policies , is also feasible inĀ (2) and attains the same objective value. This implies that our branch-and-bound tree is highly isomorphic since the scenario-based problemsĀ (6) andĀ (8) are identical (up to a permutation of the policies) across many nodes. We can reduce this undesirable symmetry by modifying StepĀ 6 of our branch-and-bound scheme as follows:
- 6ā².
Branch. Let if and let K^{\prime}=\min\Big{\{}K,\,1+\max\limits_{k\in\mathcal{K}}\big{\{}k:\Xi_{k}\neq\emptyset\big{\}}\Big{\}} otherwise. Instantiate new nodes , . Set and go to StepĀ 3.
Despite generating only a subset of the nodes that our original algorithm constructs, the modification above always maintains at least one of the solutions symmetric to every feasible solution.
Integration into MILP Solvers.
StepĀ 4 of our algorithm solves the scenario-based problemĀ (6) from scratch in every node, despite the fact that two successive problems along any branch of the branch-and-bound tree differ only by the addition of a few constraints. We can leverage this commonality if we integrate our branch-and-bound algorithm into the solution scheme of the MILP solver used for problemĀ (6). In doing so, we can also exploit the advanced facilities commonly present in the state-of-the-art solvers such as warm-starts and cutting planes, among others.
In order to integrate our branch-and-bound algorithm into the solution scheme of the MILP solver, we initialize the solver with the scenario-based problemĀ (6) corresponding to the root node of our algorithm, see StepĀ 1. The solver then proceeds to solve this problem using its own branch-and-bound procedure. Whenever the solver encounters an integral solution , we solve the associated separation problemĀ (8). If , then we execute StepĀ 6 of our algorithm through a branch callback: we report the new branches to the solver, which will discard the current solution. Otherwise, if , then we do not create any new branches, and the solver will accept as the new incumbent solution. This ensures that only those solutions which are feasible in problemĀ (5) are accepted as incumbent solutions.
Whenever the solver encounters a fractional solution, it will by default branch on an integer variable that is fractional in the current solution. However, if , it is possible to override this strategy and instead execute StepĀ 6 of our algorithm. In such cases, a heuristic rule can be used to decide whether to branch on integer variables or to branch as in StepĀ 6. In our computational experience, a simple rule that alternates between the default branching rule of the solver and the one defined by StepĀ 6 appears to perform well in practice.
3.4 Modification as a Heuristic Algorithm
Whenever the number of policies is large, the solution of the scenario-based -adaptability problemĀ (6) can be time consuming. In such cases, only a limited number of nodes will be explored by the algorithm in a given amount of computation time, and the quality of the final incumbent solution may be poor. As a remedy, we can reduce the size and complexity of the scenario-based -adaptability problemĀ (6) by fixing some of its second-stage policies. In doing so, we obtain a heuristic variant of our algorithm that can scale to large values of .
In our computational experience, a simple heuristic that sequentially solves the -, -, ā¦, -adaptability problems by fixing in each -adaptability problem all but one of the second-stage policies, , to their corresponding values in the -adaptability problem, performs well in practice. This heuristic is motivated by two observations. First, the resulting scenario-based -adaptability problemsĀ (6) have the same size and complexity as the corresponding scenario-based -adaptability problems. Second, in our experiments on instances with uncertain objective coefficients , we often found that some optimal second-stage policies of the -adaptability problem also appear in the optimal solution of the -adaptability problem. In fact, it can be shown that this heuristic can obtain -adaptable solutions that improve upon -adaptable solutions only if the objective coefficients are affected by uncertainty.
4 Numerical Results
We now analyze the computational performance of our branch-and-bound scheme in a variety of problem instances from the literature. We consider a shortest path problem with uncertain arc weights (SectionĀ 4.1), a capital budgeting problem with uncertain cash flows (SectionĀ 4.2), a variant of the capital budgeting problem with the additional option to take loans (SectionĀ 4.3), a project management problem with uncertain task durations (SectionĀ 4.4), and a vehicle routing problem with uncertain travel times (SectionĀ 4.5). Of these, the first two problems involve only binary decisions, and they can therefore also be solved with the approach described inĀ [25]. In these cases, we show that our solution scheme is highly competitive, and it frequently outperforms the approach ofĀ [25]. In contrast, the third and fourth problems also involve continuous decisions, and there is no existing solution approach for their associated -adaptability problems. However, the project management problem from SectionĀ 4.4 involves only continuous second-stage decisions, and therefore the corresponding two-stage robust optimization problemĀ (1) can also be approximated using affine decision rulesĀ [3], which represent the most popular approach for such problems. In this case, we elucidate the benefits of -adaptable constant and affine decisions over standard affine decision rules. Finally, the first and last problems involve only binary second-stage decisions and deterministic constraints, and they can therefore also be addressed with the heuristic approach described inĀ [13]. In these cases, we show that the heuristic variant of our algorithm often outperforms the latter approach in terms of solution quality.
For each problem category, we investigate the tradeoffs between computational effort and improvement in objective value of the -adaptability problem for increasing values of . We demonstrate that (i) the -adaptability problem can provide significant improvements over static robust optimization (which corresponds to the case ), and that (ii) our solution scheme can quickly determine feasible solutions of high quality.
We implemented our branch-and-bound algorithm in C++ using the CĀ callable library of CPLEXĀ 12.7Ā [26]. We used a constraint feasibility tolerance of to accept any incumbent solutions, whereas all other solver options were kept at their default values. The experiments were conducted on a single core of an IntelĀ XeonĀ 2.8GHz computer with 16GBĀ RAM.
4.1 Shortest Paths
We consider the shortest path problem fromĀ [25]. Let be a directed graph with nodes , arcs and arc weights , . Here, represents the nominal weight of the arc and denotes the uncertain deviation from the nominal weight. The realizations of the uncertain vector are known to belong to the set
[TABLE]
which stipulates that at most arc weights may maximally deviate from their nominal values.
Let and , , denote the source and terminal nodes of , respectively. The decision-maker aims to choose paths from to here-and-now, i.e., before observing the actual arc weights, such that the worst-case weight of the shortest among the chosen paths is minimized. This problem can be formulated as an instance of the -adaptability problemĀ (2):
[TABLE]
Here, denotes the set of all paths in ; that is,
[TABLE]
Note that this problem only contains second-stage decisions and as such, the corresponding two-stage robust optimization problemĀ (1) may be of limited interest in practice. Nevertheless, the -adaptability problemĀ (2) has important applications in logistics and disaster reliefĀ [25].
For each graph size , we randomly generate 100 problem instances as follows. We assign the coordinates to each node uniformly at random from the square . The nominal weight of the arc is defined to be the Euclidean distance between the nodes and ; that is, . The source node and the terminal node are defined to be the nodes with the maximum Euclidean distance between them. The arc set is obtained by removing from the set of all pairwise links the connections with the largest nominal weights. We set the uncertainty budget to . Further details on the parameter settings can be found inĀ [25].
TableĀ 2 summarizes the numerical performance of our branch-and-bound scheme for . TableĀ 2 indicates that our scheme is able to reliably compute optimal solutions for small values of and , while the average optimality gap for large values of and is less than 9%. The numerical performance is strongly affected by the value of ; very few of the -adaptable instances are solved to optimality within the time limit. This decrease in tractability is partly explained in FigureĀ 4, which shows the improvement in objective value of the -adaptability problem over the static problem (where ). FigureĀ 4(a) shows that the computed -adaptable solutions are typically of high quality since they improve upon the static solutions by as much as 13% for large values of . Moreover, FigureĀ 4(b) shows that these solutions are obtained within 1 minuteĀ (on average), even for the largest instances. This indicates that the gaps in TableĀ 2 are likely to be very conservative since the majority of computation time is spent on obtaining a certificate of optimality for these solutions.
FigureĀ 5 illustrates the quality of the solutions obtained using the heuristic variant of our algorithm, described in SectionĀ 3.3, and contrasts it with the quality of the solutions obtained using the heuristic algorithm described inĀ [13]. FigureĀ 5 shows that, after just one minute of computation time, the -, - and -adaptable solutions obtained using our heuristic algorithm are within 0.3% of known optimal solutions and about 2% better than those obtained using the heuristic algorithm described inĀ [13], on average. The differences in the qualities of the -, - and -adaptable solutions are smaller. The figure also shows that the marginal gain in objective value decreases rapidly as we increase the number of policies . Indeed, while the -adaptable solutions are about 8.3% better than the -adaptable (i.e., static) solutions, the -adaptable solutions are only about 0.1% better than the -adaptable solutions. This may be explained by the possibility that the objective values of the corresponding -adaptable solutions are very close to the optimal value of the two-stage robust optimization problemĀ (1).
4.2 Capital Budgeting
We consider the capital budgeting problem fromĀ [25], where a company wishes to invest in a subset of projects. Each project has an uncertain cost and an uncertain profit that are governed by factor models of the form
[TABLE]
In these models, and represent the nominal cost and the nominal profit of project , respectively, while and represent the row vectors of the factor loading matrices . The realizations of the uncertain vector of risk factors belong to the uncertainty set .
The company can invest in a project either before or after observing the risk factors . In the latter case, the company generates only a fraction of the profit (reflecting a penalty for postponement) but incurs the same cost as in the case of an early investment. Given an investment budget , the capital budgeting problem can then be formulated as the following instance of the two-stage robust optimization problemĀ (1):
[TABLE]
where .
For our numerical experiments, we randomly generate 100 instances for each problem size as follows. The nominal costs are chosen uniformly at random from the hyperrectangle . We then set , and . The rows of the factor loading matrices and are sampled uniformly from the unit simplex in ; that is, the row vector is sampled from such that is satisfied for all .
TableĀ 3 summarizes the numerical performance of our branch-and-bound scheme for . TableĀ 3 demonstrates that our branch-and-bound scheme performs very well for this problem class since the majority of instances is solved to optimality for . Moreover, the optimality gaps for the unsolved instances are less than 4% for and less than 9% for on average. Additionally, FigureĀ 6 shows that even for the largest instances, high-quality incumbent solutions which significantly improve (100%) upon the static robust solutions are obtained within 1Ā minute of computation time. Our results compare favorably with those ofĀ [25] as well as those of the partition-and-bound approach for the corresponding two-stage robust optimization problem presented inĀ [6].
4.3 Capital Budgeting with Loans
We consider a generalization of the capital budgeting problem from SectionĀ 4.2 where the company can increase its investment budget by purchasing a loan from the bank at a unit cost of before the risk factors are observed as well as purchasing a loan at a unit cost of (with ) after the observation occurs. If the company does not purchase any loan, then the problem reduces to the one described in SectionĀ 4.2. Therefore, we expect the worst-case profits to be at least as large as in that setting. The generalized capital budgeting problem can be formulated as the following instance of problemĀ (1):
[TABLE]
Here, . The constraint ensures that the first-stage expenditures are fully covered by the budget as well as the loan taken here-and-now.
We consider problems with projects. For each value of , we solve the same 100 instances from SectionĀ 4.2 with and . TableĀ 4 shows the computational performance of our branch-and-bound scheme for . As in the case of the problems discussed so far, the numerical tractability of our algorithm decreases as the value of increases. However, a comparison of TablesĀ 3 andĀ 4 suggests that the numerical tractability is not significantly affected by the presence of the additional continuous variables and . Indeed, the majority of instances for are solved to optimality and the average gap across all unsolved instances is less than 5% for and less than 9% for . FigureĀ 7 shows that the -adaptable solutions improve upon the static solutions by as much as 115% in the largest instances. Although not shown in the figure, a comparison of the objective values of the final incumbent solutions with those of the capital budgeting problem without loans (SectionĀ 4.2) reveals that for , the option to purchase loans has no effect on the worst-case profit of the static solution and results in less than 1% improvement in the worst-case profit of the -adaptable solution. Indeed, the option to purchase loans results in significantly better worst-case profits only if . The average relative gain in objective value is 4.3% for and 5.9% for .
4.4 Project Management
We define a project as a directed acyclic graph whose nodes represent the tasks (e.g., ābuild foundationsā or ādevelop prototypeā) and whose arcs denote the temporal precedences, i.e., implies that task cannot be started before task has been completed. We assume that each task has an uncertain duration that depends on the realization of an uncertain parameter vector . Without loss of generality, we stipulate that the project graph has the unique sink , and that the last task has a duration of zero. This can always be achieved by introducing dummy nodes and/or arcs.
In the following, we want to calculate the worst-case makespan of the project, i.e, the smallest amount of time that is required to complete the project under the worst realization of the parameter vector . This problem can be cast as the following instance of problemĀ (1):
[TABLE]
Here , and denotes the start time of task , . This problem is known to be NP-hard [37, TheoremĀ 2.1], and we will employ affine decision rules as well as -adaptable constant and affine decisions to approximate the optimal value of this problem. Note that the problem does not contain any first-stage decisions, but such decisions could be readily included, for example, to allow for resource allocations that affect the task durations.
For our numerical experiments, we consider the instance class presented inĀ [37, ExampleĀ 2.2]. To this end, we set and , and , , as well as , . FigureĀ 8 illustrates the project network corresponding to . Similar toĀ [37], we consider the uncertainty set .
We consider project networks of size . One can show that for each network size , the optimal value of the corresponding static robust optimization problem as well as the affine decision rule problem is , seeĀ [37, ExampleĀ 2.2]. For , FigureĀ 9 summarizes the computational performance of the branch-and-bound scheme and the improvement in objective value of the resulting piecewise constant and piecewise affine decision rules with pieces over the corresponding -adaptable solutions. FiguresĀ 9(a) andĀ 9(b) show that using only two pieces, piecewise constant decision rules can improve upon the affine approximation by more than 12%, while a piecewise affine decision rule can improve by more than 15%. FiguresĀ 9(c) andĀ 9(d) show that piecewise constant decision rules require smaller computation times than piecewise affine decision rules. This is not surprising since piecewise constant decision rules are parameterized by variables, whereas piecewise affine decision rules are parameterized by variables.
4.5 Vehicle Routing
We consider the classical capacitated vehicle routing problemĀ [22, 23, 35] defined on a complete, undirected graph with nodes and edges . Node [math] represents the unique depot, while each node corresponds to a customer with demand . The depot is equipped with homogeneous vehicles; each vehicle has capacity and it incurs an uncertain travel time when it traverses the edge . Here, represents the nominal travel time along the edge , while denotes the uncertain deviation from the nominal value. Similar to the shortest path problem from SectionĀ 4.1, the realizations of the uncertain vector are known to belong to the set
[TABLE]
which stipulates that at most travel times may maximally deviate from their nominal values.
A route plan corresponds to a partition of the customer set into vehicle routes, , where represents the customer and the number of customers served by the vehicle. This route plan is feasible if the total demand served on each route is less than the vehicle capacity; that is, if is satisfied for all . The total travel time of a feasible route plan under the uncertainty realization is given by , where we define ; that is, each vehicle starts and ends at the depot. The decision-maker aims to choose route plans here-and-now, i.e., before observing the actual travel times, such that the worst-case total travel time of the shortest among the chosen route plans is minimized. This problem can be formulated as an instance of the -adaptability problemĀ (2):
[TABLE]
Here, denotes the set of all feasible route plans in ; that is,
[TABLE]
Similar to the shortest path problem, the -adaptability formulation of the vehicle routing problem only contains second-stage decisions, and as such, the corresponding two-stage robust optimization problemĀ (1) is of limited interest in practice. However, the -adaptability problemĀ (2) has important applications in logistics enterprises, where the time available between observing the travel times in a road network and determining the route plan is limited, or because the drivers must be trained to a small set of route plans that are to be executed daily over the course of a year.
We note that the set represents the so-called two-index vehicle flow formulation of the vehicle routing problem, in which the first equation ensures that vehicles are used; the second set of equations ensures that each customer is visited by exactly one vehicle; while the third set of inequalities ensure that there are no subtours disconnected from the depot and that all vehicle capacities are respected. This formulation is known to be extremely challenging to solve because it consists of an exponential number of inequalities. For , the corresponding -adaptability problem is naturally even more challenging and it is practically intractable to solve it using the approach described inĀ [25]. In contrast, the heuristic variant of our algorithm described in SectionĀ 3.3 as well as the heuristic approach ofĀ [13] only require the solution of vehicle routing subproblems that are of similar complexity as the associated -adaptability problems. Therefore, in the following, we only present results using these algorithms. In both cases, we solved all vehicle routing subproblems using the branch-and-cut algorithm described inĀ [30].
For our numerical experiments, we consider all 49 instances fromĀ [30] with , which are commonly used to benchmark vehicle routing algorithms. We set an overall time limit of 2Ā hours. For the heuristic variant of our algorithm, we further set a time limit of 10Ā minutes per vehicle routing subproblem. We note that the heuristic ofĀ [13] requires the successful termination of an expensive preprocessing step to determine good -adaptable solutions. Therefore, to prevent bias in favor of our algorithm, FigureĀ 10 compares the two algorithms only across the 39 instances for which this step terminated successfully. The figure shows that when the number of policies is small, the -adaptable solutions obtained using our algorithm are about 1% better than those obtained using the heuristic algorithm ofĀ [13]. Moreover, the differences in their objective values are relatively higher for larger instances.
5 Conclusions
In contrast to single-stage robust optimization problems, which are typically solved via monolithic reformulations, there is growing evidence that two-stage and multi-stage robust optimization problems are best solved algorithmicallyĀ [6, 7, 8, 32, 39]. Our findings in this paper appear to confirm this observation, as our proposed branch-and-bound algorithm compares favorably with the reformulations proposed inĀ [25]. In terms of modeling flexibility, our algorithm can accommodate mixed continuous and discrete decisions in both stages, can incorporate discrete uncertainty, and allows us to model flexible piecewise affine decision rules. At the same time, our numerical results indicate that the algorithmic approach is highly competitive in terms of computational performance as well. From a practical viewpoint, a notable feature of our algorithm is that it admits a lightweight implementation by integrating it into the branch-and-bound schemes of commercial solvers via branch callbacks, while allowing easy modification as a heuristic for large-scale instances.
Our results open up multiple fruitful avenues for future research. The scope of the presented algorithm can be broadened further by generalizing it to two-stage distributionally robust optimization problems, where the uncertain problem parameters are modeled as random variables following a probability distribution that is only partially known. More ambitiously, it would be instructive to explore how the concept of -adaptability, as well as our proposed branch-and-bound algorithm, extend to dynamic robust optimization problems with more than two stages.
Acknowledgments
Anirudh Subramanyam and Chrysanthos E.Ā Gounaris gratefully acknowledge support from the National Science Foundation, award number CMMI-1434682. Wolfram Wiesemann gratefully acknowledges funding from the EPSRC grants EP/M028240/1, EP/M027856/1 and EP/N020030/1. Anirudh Subramanyam also acknowledges support from the John and Claire Bertucci Graduate Fellowship program at Carnegie Mellon University.
Appendix A Analysis of the Two-Stage Robust Optimization Problem
To analyze problemĀ (1), we equivalently rewrite it as
[TABLE]
for and .
We first investigate whether the infima and the supremum in problemĀ (1*ā²*) are attained.
Proposition A.1** (Continuity).**
ProblemĀ (1*ā²*) satisfies the following properties.
- (i)
The problem attains its infimum, if it is feasible. 2. (ii)
The problem attains its supremum, if only the objective function is uncertain. Otherwise, does not necessarily attain its supremum, even if only the constraint right-hand sides are uncertain. 3. (iii)
The problemĀ (1ā²**) attains its infimum, if it is feasible.
Proof.
The first statement holds since the problem minimizes an affine function in over the intersection of the compact set and the polyhedron .
In view of the second statement, assume first that only the objective function in problemĀ (1*ā²*) is uncertain; that is, , and for all . Then the inner problem simplifies to with . If , then for all , and any attains the supremum. Otherwise, we have , where denotes the set of (finitely many) extreme points of the convex hull of . We thus conclude fromĀ [33, PropositionĀ 1.26] that is upper semicontinuous in for every fixed , andĀ [33, TheoremĀ 1.9] then implies that attains its supremum since is a compact set.
Assume now that only the constraint right-hand sides in problemĀ (1*ā²*) are allowed to be uncertain, and consider the following instance of problem :
[TABLE]
The inner minimization problem is optimized by and . The supremum is , and it is approached by the sequence of feasible solutions . Note, however, that the limit point of this sequence results in an objective function value of [math].
As for the third statement, we first note that the extended real-valued function
[TABLE]
is lower semicontinuous in since the set is closed. FromĀ [33, TheoremĀ 1.17] andĀ [33, PropositionĀ 1.26] we conclude that the lower semicontinuity is preserved by the partial minimization over and the partial maximization over . ByĀ [33, TheoremĀ 1.9] and the fact that is compact we can then conclude that the problemĀ (1*ā²*) attains its infimum whenever it is feasible. ā
It is shown inĀ [25, ExampleĀ 1] that may not attain its supremum if both the objective function and the constraint right-hand sides in problemĀ (1*ā²*) are uncertain. PropositionĀ A.1Ā (ii) strengthens this observation to instances of problemĀ (1*ā²*) where only the constraint right-hand sides are uncertain. We now consider the convexity properties of problemĀ (1*ā²*).
Proposition A.2** (Convexity).**
ProblemĀ (1*ā²*) satisfies the following properties.
- (i)
The problem is convex, if is convex. 2. (ii)
The problem is convex, if is convex and only the objective function is uncertain, irrespective of . Otherwise, is typically not convex, even if and are convex and only the constraint right-hand sides are uncertain. 3. (iii)
The problemĀ (1ā²**) is convex, if and are convex, irrespective of .
Proof.
The first statement directly follows from the linearity of the objective function and the convexity of the feasible set.
In view of the second statement, assume first that is convex and only the objective function in problemĀ (1*ā²*) is uncertain; that is, , and for all . Then the inner problem simplifies to with . Assume that ; the other case is trivial. Then we have , where denotes the set of (finitely many) extreme points of the convex hull of . We thus conclude fromĀ [33, PropositionĀ 2.9] that is concave in for every fixed , which implies that is a convex optimization problem.
Assume now that the constraint right-hand sides in problemĀ (1*ā²*) are allowed to be uncertain, and consider the following instance of problem :
[TABLE]
Since the inner minimization problem is optimized by , the problem maximizes the (convex) -norm of over the interval , which amounts to a non-convex optimization problem.
As for the third statement, assume that and are convex, and consider the function
[TABLE]
The function is convex in for every fixed . FromĀ [33, PropositionĀ 2.22] andĀ [33, PropositionĀ 2.9] we conclude that the convexity is preserved by the partial minimization over and the partial maximization over . The problemĀ (1*ā²*) thus minimizes the sum of two convex functions and over the convex set , which is a convex optimization problem. ā
One readily verifies that the third statement in PropositionĀ A.2 does not hold if is not convex. We now investigate under which conditions we can solve problemĀ (1*ā²*) in polynomial time.
Proposition A.3** (Tractability).**
ProblemĀ (1*ā²*) satisfies the following properties.
- (i)
The problem can be solved in polynomial time, if is convex. 2. (ii)
The problem can be solved in polynomial time, if and are convex and only the objective function is uncertain. Otherwise, is strongly NP-hard, even if and are convex and only the constraint right-hand sides are uncertain. 3. (iii)
The problemĀ (1ā²**) can be solved in polynomial time, if , and are convex and only the objective function is uncertain. Otherwise, the problem is strongly NP-hard, even if , and are convex and only the constraint right-hand sides are uncertain.
Proof.
If is convex, then the problem amounts to a linear program that can be solved in polynomial time. This shows the first statement.
The first part of the second statement follows from the proof of the first part of the third statement below if we fix and in problemĀ (1*ā²*). For the second part of the second statement, we recall the strongly NP-hard 0/1 Integer Programming (IP) feasibility problemĀ [17]:
0/1 Integer Programming Feasibility.
aĀ Ā Ā Instance. Given are and .
aĀ Ā Ā Question. Is there a vector such that ?
We show that the IP feasibility problem has an affirmative answer if and only if the problem
[TABLE]
has an optimal value of . Note thatĀ (9) can be interpreted as an instance of . For any fixed , the infimum inĀ (9) evaluates to . Thus, the optimal value ofĀ (9) is equal to if and only if there is satisfying and . The statement now follows from the fact that if and only if .
In view of the first part of the third statement, the proof of PropositionĀ A.4 below shows that, under the stated assumptions, problemĀ (1*ā²*) reduces to a single-stage robust optimization problem. Dualizing the inner maximization problem in that single-stage reformulation allows us to solve the overall problem in polynomial time as a linear program, see [2]. Finally, the second part of the third statement follows from the proof of the second part of the second statement if we amend problemĀ (9) by appending an outer infimum over the singleton set . ā
The NP-hardness of the two-stage robust optimization problemĀ (1*ā²*) has previously been established for the case where , and are convex and only the constraint right-hand sides are uncertain [24, TheoremĀ 3.5]. We provide an alternative proof in PropositionĀ A.3Ā (iii) to facilitate a self-contained comparison with the -adaptability problemĀ (2) in PropositionĀ B.3Ā (ii) below. Moreover, it is shown inĀ [24, TheoremĀ 3.3] that problemĀ (1*ā²*) can be solved in polynomial time whenever and are convex, and are deterministic and is described in terms of its extreme points. We close our analysis of the two-stage robust optimization problemĀ (1*ā²*) with a special case where it reduces to a single-stage robust optimization problem.
Proposition A.4** (Reduction to Static Problem).**
The problemĀ (1*ā²*) reduces to a static robust optimization problem where is replaced with its convex hull, if is convex and only the objective function is uncertain, irrespective of and .
Proof.
Assume that is convex and that only the objective function in problemĀ (1*ā²*) is uncertain; that is, , and for all . ProblemĀ (1*ā²*) then simplifies to
[TABLE]
where the replacement of the second-stage feasible region with its convex hull, , is justified since the inner minimization has a linear objective function. The classical minimax theorem now allows us to exchange the order of the inner two operators:
[TABLE]
This problem is readily recognized as a single-stage robust optimization problem.
We emphasize that the previous argument requires to be convex. Indeed, we have
[TABLE]
and we cannot establish equivalence by replacing with in the second optimization problem either. ā
A related result was established inĀ [3, TheoremĀ 2.1], where it is shown that the two-stage robust optimization problemĀ (1*ā²*) reduces to a single-stage robust optimization problem if , and are convex and the uncertain parameters can be partitioned into subsets such that each constraint is only affected by the parameters of one such subset, and no two constraints are affected by parameters of the same subset.
Appendix B Analysis of the -Adaptability Problem
To analyze problemĀ (2), we equivalently rewrite it as
[TABLE]
for and .
In analogy to PropositionĀ A.1, we first investigate whetherĀ (2*ā²*) attains its infima and supremum.
Proposition B.1** (Continuity).**
ProblemĀ (2*ā²*) satisfies the following properties.
- (i)
The problem attains its supremum, if only the objective function or only the constraints are uncertain. Otherwise, does not necessarily attain its supremum, even if the constraint left-hand sides are not uncertain. 2. (ii)
The problemĀ (2ā²**) attains its infimum, if it is feasible.
Proof.
In view of the first statement, assume that only the objective function in problemĀ (2*ā²*) is uncertain; that is, , and for all . Then the inner problem simplifies to , where . If , then for all , and any attains the supremum. Otherwise, is readily verified to be continuous in for every fixed and , andĀ [33, TheoremĀ 1.9] then implies that attains its supremum since is a compact set.
Assume now that only the constraints in problemĀ (2*ā²*) are uncertain. Then simplifies to , where . If , then for any and attains its supremum. Otherwise, , and the supremum in is attained by any , where
[TABLE]
Finally, assume that both the objective function and the constraint right-hand sides in problemĀ (2*ā²*) are allowed to be uncertain, and consider the following instance of problem :
[TABLE]
The inner minimization problem is optimized by if and otherwise. The supremum is , and it is approached by the sequence of feasible solutions . Note, however, that the limit point of this sequence results in an objective function value of [math].
As for the second statement, we first note that each extended real-valued function
[TABLE]
is lower semicontinuous in since the sets are closed. FromĀ [33, PropositionĀ 1.26] we conclude that the lower semicontinuity is preserved by the partial minimization over and the partial maximization over . Due toĀ [33, TheoremĀ 1.9] the problemĀ (2*ā²*) then attains its infimum whenever it is feasible. ā
Similar to PropositionĀ A.2, we now consider the convexity properties of problemĀ (2*ā²*).
Proposition B.2** (Convexity).**
ProblemĀ (2*ā²*) satisfies the following properties.
- (i)
The problem is convex, if is convex and only the objective function is uncertain. Otherwise, is typically not convex, even if is convex and only the constraint right-hand sides are uncertain. 2. (ii)
ProblemĀ (2ā²**) is typically not convex, even if and are convex and is a singleton.
Proof.
In view of the first statement, assume that is convex and that only the objective function in problemĀ (2*ā²*) is uncertain; that is, , and for all . Then the inner problem simplifies to , where . Assume that ; the other case is trivial. Then is a piecewise-affine concave function in for every fixed and . We thus conclude that is a convex optimization problem as it maximizes a concave function over a convex set.
Assume now that the constraint right-hand sides in problemĀ (2*ā²*) are allowed to be uncertain, and consider the following instance of problem :
[TABLE]
The inner minimization problem is optimized by , if , and , otherwise. The outer maximization problem thus maximizes the non-convex function over the interval , which amounts to a non-convex optimization problem.
In view of the second statement, consider the following problem instance:
[TABLE]
The problem attains the objective value for , but it attains the larger objective value of [math] for . ā
In analogy to PropositionĀ A.3, we now investigate under which conditions problemĀ (2*ā²*) is tractable.
Proposition B.3** (Tractability).**
ProblemĀ (2*ā²*) satisfies the following properties.
- (i)
The problem can be solved in polynomial time, if is convex and only the objective function is uncertain. Otherwise, is strongly NP-hard, even if is convex and only the constraint right-hand sides are uncertain. 2. (ii)
The problemĀ (2ā²**) can be solved in polynomial time, if , and are convex and only the objective function is uncertain. Otherwise, the problem is strongly NP-hard, even if , and are convex and only the constraint right-hand sides are uncertain.
Proof.
The first part of the first statement follows directly from [25, ObservationĀ 2].
In view of the second part of the first statement, we consider the following variant of the IP feasibility problem:
Approximate 0/1 Integer Programming Feasibility.
aĀ Ā Ā Instance. Given are , .
aĀ Ā Ā Question. Is there , \epsilon=\big{(}\min_{r}\sum_{q}|A_{rq}|\big{)}^{-1}, such that ?
It follows from [38, LemmaĀ 2] that the approximate IP feasibility problem is strongly NP-hard. We claim that the approximate IP feasibility problem has an affirmative answer if and only if
[TABLE]
where for from the approximate IP feasibility instance, , for and . Note that we do not specify the first-stage decision as it is not required for the argument.
Assume first that the approximate IP feasibility problem has an affirmative answer, i.e., there is such that . Note that , but for every we have . Since , is the only feasible candidate policy, and the optimal value of problemĀ (10) is indeed .
Assume now that the optimal value of problemĀ (10) is . In that case, there is such that , i.e, . Since by construction of , we conclude that the approximate IP feasibility problem has an affirmative answer.
As for the first part of the second statement, assume first that , and are convex and that only the objective function in problemĀ (2*ā²*) is uncertain; that is, , and for all . The classical minimax theorem then implies that
[TABLE]
i.e., the -adaptability problem achieves the same objective value as the two-stage robust optimization problemĀ (1). Since the -adaptability problem is bounded from above by the -adaptability problem and from below by the two-stage robust optimization problem, we thus conclude that, under the stated assumptions, the -adaptability problem coincides with the -adaptability problem. The statement now follows from the fact that we can reformulate the -adaptability problem as a linear program by dualizing the inner maximization problem, seeĀ [2]. Note that the convexity of is needed in this argument since
[TABLE]
A similar argument with and shows that the convexity of is needed, too.
Finally, in view of the second part of the second statement, consider the following problem:
[TABLE]
where , , , and for from an approximate IP feasibility instance. We claim that the optimal value of this problem is , i.e., there is for which no decision is feasible in the second stage, if and only if the approximate IP feasibility instance has an affirmative answer.
Assume first that the approximate IP feasibility problem has an affirmative answer. It then follows from [38, LemmaĀ 2] that the exact IP feasibility problem also has an affirmative answer; that is, there is such that . Fix any feasible candidate policy . By construction, there is such that . Thus, the candidate policy is feasible in the second stage under the parameter realization only if , which is not the case since . We thus conclude that the optimal value of problemĀ (11) is indeed whenever the approximate IP feasibility problem has an affirmative answer.
Assume now that the optimal value of problemĀ (11) is and consider the candidate policies , . Since and problemĀ (11) evaluates to under this choice of , there must be such that ; that is, . Since by construction of , we conclude that the approximate IP feasibility problem has an affirmative answer. ā
We note that can be solved in polynomial time if is convex and the number of policies is fixed (even when the objective function, the constraint coefficients and the right-hand sides are uncertain), see [25, CorollaryĀ 1]. The NP-hardness of has previously been established under the more restrictive assumption that both the objective function and the constraint right-hand sides in problemĀ (2*ā²*) are uncertain, see [25, TheoremĀ 3]. For the special case where , the -adaptability problem with objective and constraint uncertainty can be solved in polynomial time if any of , or is fixed [5, PropositionĀ 5], while the problem becomes NP-hard otherwiseĀ [5, PropositionĀ 6]. PropositionĀ B.3Ā (ii) provides an alternative proof of the NP-hardness of problemĀ (2*ā²*), which facilitates a direct comparison to the two-stage robust optimization problemĀ (1*ā²*), see PropositionĀ A.3Ā (iii).
We also note that the -adaptability problemĀ (2*ā²*) simplifies in the absence of first-stage decisions . For this case, it has been shown in [13, TheoremĀ 2] that the problem can be solved in polynomial time, if only the objective function is uncertain, the deterministic second-stage problem is polynomial time solvable, the uncertainty set has a tractable representation, and if any number of policies is acceptable. The same problem becomes NP-hard, however, when the number of policies is fixed [13, CorollaryĀ 3] or when the uncertainty sets are discrete [14, CorollariesĀ 1ā4].
Similar to PropositionĀ A.4, we next consider when the -adaptability problemĀ (2*ā²*) reduces to a single-stage robust optimization problem.
Proposition B.4** (Reduction to Static Problem).**
The problemĀ (2*ā²*) reduces to a static robust optimization problem where is replaced with its convex hull, if is convex, only the objective function is uncertain and , irrespective of and .
Proof.
Assume that is convex and that only the objective function in problemĀ (2*ā²*) is uncertain; that is, , and for all . ProblemĀ (2*ā²*) then simplifies to
[TABLE]
The inner discrete minimization in this problem can be replaced by a continuous minimization over all convex combinations :
[TABLE]
The classical minimax theorem then allows us to exchange the order of the inner two operators:
[TABLE]
If , then we can use CarathĆ©odoryās theorem to replace the convex combinations of candidate decisions with a single decision from the convex hull, , resulting in
[TABLE]
which is readily recognized as a single-stage robust optimization problem. Similarly, problemĀ (12) can be rewritten as
[TABLE]
and if , then we can use CarathĆ©odoryās theorem to replace the convex combinations of candidate decisions with a single decision from the convex hull of their domain, , resulting again in the single-stage robust optimization problemĀ (13).
As in PropositionĀ A.4, the previous arguments require to be convex. Indeed, we have
[TABLE]
and we cannot establish equivalence by replacing with in the second optimization problem either. To see that the number of policies must exceed in general in the previous arguments, we compare the two problems
[TABLE]
For every , the problem on the right attains its optimal value [math] at any with . Similarly, for every , the problem on the left attains an optimal value of [math], if , and an optimal value of , if . We thus verify that is required for the optimal values to coincide by choosing . ā
We close this section with several special cases where the optimal value of the -adaptability problemĀ (2*ā²*) coincides with the optimal value of the two-stage robust optimization problemĀ (1*ā²*).
Proposition B.5** (Optimality).**
The optimal values of problemsĀ (1*ā²*) andĀ (2*ā²*) coincide if (i) and are convex and only the objective function is uncertain, irrespective of and , or if (ii) is convex, only the objective function is uncertain and , irrespective of and , or if (iii) has a finite cardinality and , irrespective of and . Otherwise, their optimal values may differ for any finite , even if , and are convex and only the constraint right-hand sides are uncertain.
Proof.
The fact that the two optimal values coincide under the first set of conditions follows from the proof of PropositionĀ B.3Ā (ii), where we have shown that, under the stated assumptions, the optimal values of the -adaptability problem and the two-stage robust optimization problemĀ (1*ā²*) coincide. We have also shown there that the convexity of and is crucial for the proof to hold.
The fact that the two optimal values coincide under the second set of conditions follows from PropositionĀ B.4, which shows that, under the stated assumptions, the -adaptability problemĀ (2*ā²*) is equivalent to
[TABLE]
The classical minimax theorem, which is applicable since and are convex, then implies that this problem is equivalent to
[TABLE]
which provides a lower bound to the two-stage robust optimization problemĀ (1*ā²*) since . On the other hand, we know that the -adaptability problemĀ (2*ā²*) by construction boundsĀ (1*ā²*) from above. We thus conclude that the optimal values of both problems must coincide.
In view of the third set of conditions, it is clear that policies are sufficient for the optimal values of problemsĀ (1*ā²*) andĀ (2*ā²*) to coincide. Moreover, [25, TheoremĀ 4] presents a problem where the optimal values of problemsĀ (1*ā²*) andĀ (2*ā²*) differ for every .
As for the second part of the statement, consider the problem
[TABLE]
The optimal second-stage decision to this two-stage robust optimization problem satisfies and , and any attains the optimal objective value [math]. Consider now the -adaptability problem
[TABLE]
By construction, the objective value of this problem evaluates to at least for any finite and any feasible solution . ā
The validity of the first part of the statement under the second set of conditions in PropositionĀ B.5 was established for the subclass of purely binary -adaptability problems inĀ [25, TheoremĀ 1]. We also mention that the optimal value of the -adaptability problemĀ (2*ā²*) approaches the optimal value of the two-stage robust optimization problemĀ (1*ā²*) if and a continuity assumption is satisfied, see [5, Proposition 1].
The reference list from the paper itself. Each links out to its DOI / PubMed record.
- 1[1] J. Ayoub and M. Poss. Decomposition for adjustable robust linear optimization subject to uncertainty polytope. Computational Management Science , 13(2):219ā239, 2016.
- 2[2] A. Ben-Tal, L. El Ghaoui, and A. Nemirovski. Robust Optimization . Princeton University Press, 2009.
- 3[3] A. Ben-Tal, A. Goryashko, E. Guslitzer, and A. Nemirovski. Adjustable robust solutions of uncertain linear programs. Mathematical Programming , 99(2):351ā376, 2004.
- 4[4] D. Bertsimas, D. B. Brown, and C. Caramanis. Theory and applications of robust optimization. SIAM Review , 53(3):464ā501, 2011.
- 5[5] D. Bertsimas and C. Caramanis. Finite adaptibility in multistage linear optimization. IEEE Transactions on Automatic Control , 55(12):2751ā2766, 2010.
- 6[6] D. Bertsimas and I. Dunning. Multistage robust mixed integer optimization with adaptive partitions. Operations Research , 64(4):980ā998, 2016.
- 7[7] D. Bertsimas and A. Georghiou. Design of near optimal decision rules in multistage adaptive mixed-integer optimization. Operations Research , 63(3):610ā627, 2015.
- 8[8] D. Bertsimas and A. Georghiou. Binary decision rules for multistage adaptive mixed-integer optimization. Mathematical Programming , pages 1ā39, 2017.
