Learning Branching Policies for MILPs with Proximal Policy Optimization

TL;DR

This paper introduces TGPPO, a reinforcement learning framework using PPO to learn branching policies for MILPs, which generalizes better across diverse instances compared to imitation learning methods.

Contribution

It proposes a novel RL-based approach with a dynamic state representation for branching in MILPs, improving generalization over existing imitation learning methods.

Findings

TGPPO outperforms existing policies in reducing explored nodes.

It improves p-Primal-Dual Integrals (PDI) especially on out-of-distribution instances.

The approach demonstrates robustness and adaptability in diverse MILP problems.

Abstract

Branch-and-Bound (B\&B) is the dominant exact solution method for Mixed Integer Linear Programs (MILP), yet its exponential time complexity poses significant challenges for large-scale instances. The growing capabilities of machine learning have spurred efforts to improve B\&B by learning data-driven branching policies. However, most existing approaches rely on Imitation Learning (IL), which tends to overfit to expert demonstrations and struggles to generalize to structurally diverse or unseen instances. In this work, we propose Tree-Gate Proximal Policy Optimization (TGPPO), a novel framework that employs Proximal Policy Optimization (PPO), a Reinforcement Learning (RL) algorithm, to train a branching policy aimed at improving generalization across heterogeneous MILP instances. Our approach builds on a parameterized state space representation that dynamically captures the evolving context of the search tree. Empirical evaluations show that TGPPO often outperforms existing learning-based policies in terms of reducing the number of nodes explored and improving p-Primal-Dual Integrals (PDI), particularly in out-of-distribution instances. These results highlight the potential of RL to develop robust and adaptable branching strategies for MILP solvers.