ReviBranch: Deep Reinforcement Learning for Branch-and-Bound with Revived Trajectories

TL;DR

ReviBranch is a deep reinforcement learning framework that improves branch-and-bound efficiency for MILPs by utilizing revived trajectories and dense reward signals, leading to better generalization and fewer search nodes.

Contribution

It introduces a novel deep RL approach with revived trajectories and importance-weighted rewards to enhance branching decisions in MILP solvers.

Findings

Reduces B&B nodes by 4.0% on large-scale instances

Decreases LP iterations by 2.2%

Outperforms existing RL methods in diverse benchmarks

Abstract

The Branch-and-bound (B&B) algorithm is the main solver for Mixed Integer Linear Programs (MILPs), where the selection of branching variable is essential to computational efficiency. However, traditional heuristics for branching often fail to generalize across heterogeneous problem instances, while existing learning-based methods such as imitation learning (IL) suffers from dependence on expert demonstration quality, and reinforcement learning (RL) struggles with limitations in sparse rewards and dynamic state representation challenges. To address these issues, we propose ReviBranch, a novel deep RL framework that constructs revived trajectories by reviving explicit historical correspondences between branching decisions and their corresponding graph states along search-tree paths. During training, ReviBranch enables agents to learn from complete structural evolution and temporal dependencies within the branching process. Additionally, we introduce an importance-weighted reward redistribution mechanism that transforms sparse terminal rewards into dense stepwise feedback, addressing the sparse reward challenge. Extensive experiments on different MILP benchmarks demonstrate that ReviBranch outperforms state-of-the-art RL methods, reducing B&B nodes by 4.0% and LP iterations by 2.2% on large-scale instances. The results highlight the robustness and generalizability of ReviBranch across heterogeneous MILP problem classes.