Solving NP-Hard Problems on Graphs with Extended AlphaGo Zero

TL;DR

This paper introduces a novel reinforcement learning framework inspired by AlphaGo Zero to solve NP-hard graph problems, demonstrating improved generalization and performance over previous methods like S2V-DQN.

Contribution

The authors adapt AlphaGo Zero's learning strategy to combinatorial optimization on graphs, enhancing generalization and integrating advanced GNN models for better results.

Findings

Outperforms S2V-DQN on various NP-hard problems

Generalizes better to different graph structures

Combining with GIN improves performance

Abstract

There have been increasing challenges to solve combinatorial optimization problems by machine learning. Khalil et al. proposed an end-to-end reinforcement learning framework, S2V-DQN, which automatically learns graph embeddings to construct solutions to a wide range of problems. To improve the generalization ability of their Q-learning method, we propose a novel learning strategy based on AlphaGo Zero which is a Go engine that achieved a superhuman level without the domain knowledge of the game. Our framework is redesigned for combinatorial problems, where the final reward might take any real number instead of a binary response, win/lose. In experiments conducted for five kinds of NP-hard problems including {\sc MinimumVertexCover} and {\sc MaxCut}, our method is shown to generalize better to various graphs than S2V-DQN. Furthermore, our method can be combined with recently-developed graph neural network (GNN) models such as the \emph{Graph Isomorphism Network}, resulting in even better performance. This experiment also gives an interesting insight into a suitable choice of GNN models for each task.