A Graph Neural Network-Based QUBO-Formulated Hamiltonian-Inspired Loss Function for Combinatorial Optimization using Reinforcement Learning

TL;DR

This paper introduces a novel GNN-based QUBO Hamiltonian-inspired loss function for combinatorial optimization and demonstrates its effectiveness within a reinforcement learning framework, achieving significant performance improvements.

Contribution

It proposes integrating a QUBO-formulated Hamiltonian as a reward function in reinforcement learning for graph-based combinatorial optimization, enhancing existing GNN approaches.

Findings

Up to 44% improvement over PI-GNN in experiments

Effective integration of QUBO Hamiltonian as RL reward function

Enhanced performance on dense graph instances

Abstract

Quadratic Unconstrained Binary Optimization (QUBO) is a generic technique to model various NP-hard combinatorial optimization problems in the form of binary variables. The Hamiltonian function is often used to formulate QUBO problems where it is used as the objective function in the context of optimization. Recently, PI-GNN, a generic scalable framework, has been proposed to address the Combinatorial Optimization (CO) problems over graphs based on a simple Graph Neural Network (GNN) architecture. Their novel contribution was a generic QUBO-formulated Hamiltonian-inspired loss function that was optimized using GNN. In this study, we address a crucial issue related to the aforementioned setup especially observed in denser graphs. The reinforcement learning-based paradigm has also been widely used to address numerous CO problems. Here we also formulate and empirically evaluate the compatibility of the QUBO-formulated Hamiltonian as the generic reward function in the Reinforcement Learning paradigm to directly integrate the actual node projection status during training as the form of rewards. In our experiments, we observed up to 44% improvement in the RL-based setup compared to the PI-GNN algorithm. Our implementation can be found in https://github.com/rizveeredwan/learning-graph-structure.