Learning Heuristics over Large Graphs via Deep Reinforcement Learning

TL;DR

This paper introduces GCOMB, a scalable deep reinforcement learning framework that learns heuristics for large-scale graph problems, achieving significant speedups with comparable or better solution quality.

Contribution

GCOMB combines GCNs and Q-learning with importance sampling to efficiently learn heuristics for billion-sized graphs, addressing scalability and budget constraints.

Findings

GCOMB is 100 times faster than existing algorithms.

GCOMB achieves marginally better solution quality.

GCOMB is 150 times faster on Influence Maximization.

Abstract

There has been an increased interest in discovering heuristics for combinatorial problems on graphs through machine learning. While existing techniques have primarily focused on obtaining high-quality solutions, scalability to billion-sized graphs has not been adequately addressed. In addition, the impact of budget-constraint, which is necessary for many practical scenarios, remains to be studied. In this paper, we propose a framework called GCOMB to bridge these gaps. GCOMB trains a Graph Convolutional Network (GCN) using a novel probabilistic greedy mechanism to predict the quality of a node. To further facilitate the combinatorial nature of the problem, GCOMB utilizes a Q-learning framework, which is made efficient through importance sampling. We perform extensive experiments on real graphs to benchmark the efficiency and efficacy of GCOMB. Our results establish that GCOMB is 100 times faster and marginally better in quality than state-of-the-art algorithms for learning combinatorial algorithms. Additionally, a case-study on the practical combinatorial problem of Influence Maximization (IM) shows GCOMB is 150 times faster than the specialized IM algorithm IMM with similar quality.