Projecting Latent RL Actions: Towards Generalizable and Scalable Graph Combinatorial Optimization

TL;DR

This paper introduces projection agents, a novel RL-GCO method that operates in a continuous latent space for scalable, generalizable graph combinatorial optimization, achieving faster inference and better generalization.

Contribution

The authors propose a new RL-GCO approach using latent action spaces, enabling scalable, generalizable solutions and providing a Python library for reproducibility.

Findings

Achieves up to 16.2x faster inference

Improves generalization by up to 40%

Supports super-linear decision spaces with interdependent variables

Abstract

Graph combinatorial optimization (GCO) has attracted growing interest, as many NP-hard problems naturally admit graph formulations, yet their combinatorial explosion renders exact methods computationally intractable. Recent advances in Reinforcement Learning (RL) combined with Graph Neural Networks (GNNs) have significantly improved learning-based GCO solvers. However, existing approaches face limitations in both generalization across diverse graph instances and computational scalability as action spaces grow. To address both challenges, we introduce projection agents, a novel RL-GCO approach that operates directly in a continuous GNN-based action embedding space, predicting a desired latent action in a single forward pass and subsequently decoding it into a valid discrete action. Additionally, we enable fair comparison across RL methods through a shared embedding space for both observations and actions. Across diverse benchmarks, our approach achieves up to 16.2x faster inference and up to 40% better generalization than existing solutions using only simple nearest-neighbor decoding, while opening the door to strong RL performance in super-linear decision spaces with multiple interdependent variables. Finally, we release LaGCO-RL, a Python library that automates latent action-space construction and supports existing RL-GCO solutions, promoting reproducibility and adaptation to new GCO benchmarks.