Improving Optimization Bounds using Machine Learning: Decision Diagrams meet Deep Reinforcement Learning

TL;DR

This paper introduces a novel deep reinforcement learning method to optimize variable orderings in decision diagrams, significantly improving bounds for discrete optimization problems like Max Independent Set and Max Cut.

Contribution

It is the first to apply machine learning to enhance relaxation bounds in decision diagrams for combinatorial optimization problems.

Findings

Deep reinforcement learning yields tighter bounds than traditional methods.

The approach outperforms existing ordering heuristics on synthetic instances.

Tighter bounds lead to better approximation of optimal solutions.

Abstract

Finding tight bounds on the optimal solution is a critical element of practical solution methods for discrete optimization problems. In the last decade, decision diagrams (DDs) have brought a new perspective on obtaining upper and lower bounds that can be significantly better than classical bounding mechanisms, such as linear relaxations. It is well known that the quality of the bounds achieved through this flexible bounding method is highly reliant on the ordering of variables chosen for building the diagram, and finding an ordering that optimizes standard metrics is an NP-hard problem. In this paper, we propose an innovative and generic approach based on deep reinforcement learning for obtaining an ordering for tightening the bounds obtained with relaxed and restricted DDs. We apply the approach to both the Maximum Independent Set Problem and the Maximum Cut Problem. Experimental results on synthetic instances show that the deep reinforcement learning approach, by achieving tighter objective function bounds, generally outperforms ordering methods commonly used in the literature when the distribution of instances is known. To the best knowledge of the authors, this is the first paper to apply machine learning to directly improve relaxation bounds obtained by general-purpose bounding mechanisms for combinatorial optimization problems.