A Markov Decision Process for Variable Selection in Branch & Bound

TL;DR

This paper introduces BBMDP, a Markov Decision Process framework for variable selection in Branch & Bound algorithms, enabling reinforcement learning to improve MILP solving efficiency, with empirical validation showing superior performance over existing RL agents.

Contribution

The paper presents a novel MDP formulation for variable selection in B&B, facilitating the application of RL algorithms to learn better heuristics for MILP problems.

Findings

The BBMDP model outperforms previous RL agents on four MILP benchmarks.

Empirical results demonstrate improved B&B efficiency using the proposed MDP approach.

The formulation enables leveraging a broad range of RL algorithms for B&B heuristics.

Abstract

Mixed-Integer Linear Programming (MILP) is a powerful framework used to address a wide range of NP-hard combinatorial optimization problems, often solved by Branch and Bound (B&B). A key factor influencing the performance of B&B solvers is the variable selection heuristic governing branching decisions. Recent contributions have sought to adapt reinforcement learning (RL) algorithms to the B&B setting to learn optimal branching policies, through Markov Decision Processes (MDP) inspired formulations, and ad hoc convergence theorems and algorithms. In this work, we introduce BBMDP, a principled vanilla MDP formulation for variable selection in B&B, allowing to leverage a broad range of RL algorithms for the purpose of learning optimal B\&B heuristics. Computational experiments validate our model empirically, as our branching agent outperforms prior state-of-the-art RL agents on four standard MILP benchmarks.