Aligning LLMs with Graph Neural Solvers for Combinatorial Optimization

TL;DR

AlignOPT combines large language models with graph neural networks to improve the accuracy and scalability of solving combinatorial optimization problems by aligning semantic and structural representations.

Contribution

This paper introduces AlignOPT, a novel method that aligns LLMs with graph neural solvers to enhance generalization and performance in combinatorial optimization tasks.

Findings

Achieves state-of-the-art results across diverse COPs.

Demonstrates strong generalization to unseen instances.

Improves accuracy and scalability of COP solutions.

Abstract

Recent research has demonstrated the effectiveness of large language models (LLMs) in solving combinatorial optimization problems (COPs) by representing tasks and instances in natural language. However, purely language-based approaches struggle to accurately capture complex relational structures inherent in many COPs, rendering them less effective at addressing medium-sized or larger instances. To address these limitations, we propose AlignOPT, a novel approach that aligns LLMs with graph neural solvers to learn a more generalizable neural COP heuristic. Specifically, AlignOPT leverages the semantic understanding capabilities of LLMs to encode textual descriptions of COPs and their instances, while concurrently exploiting graph neural solvers to explicitly model the underlying graph structures of COP instances. Our approach facilitates a robust integration and alignment between linguistic semantics and structural representations, enabling more accurate and scalable COP solutions. Experimental results demonstrate that AlignOPT achieves state-of-the-art results across diverse COPs, underscoring its effectiveness in aligning semantic and structural representations. In particular, AlignOPT demonstrates strong generalization, effectively extending to previously unseen COP instances.