STRCMP: Integrating Graph Structural Priors with Language Models for Combinatorial Optimization

TL;DR

STRCMP introduces a structure-aware framework combining graph neural networks and large language models to improve algorithm discovery for combinatorial optimization problems, enhancing solution quality and efficiency.

Contribution

It is the first to systematically integrate structural priors into LLM-based algorithm discovery for CO problems, improving performance over existing methods.

Findings

Outperforms five strong neural and LLM-based methods in solution optimality.

Achieves significant improvements in computational efficiency.

Demonstrates effectiveness across multiple benchmark datasets.

Abstract

Combinatorial optimization (CO) problems, central to operation research and theoretical computer science, present significant computational challenges due to their NP-hard nature. While large language models (LLMs) have emerged as promising tools for CO--either by directly generating solutions or synthesizing solver-specific codes--existing approaches often neglect critical structural priors inherent to CO problems, leading to suboptimality and iterative inefficiency. Inspired by human experts' success in leveraging CO structures for algorithm design, we propose STRCMP, a novel structure-aware LLM-based algorithm discovery framework that systematically integrates structure priors to enhance solution quality and solving efficiency. Our framework combines a graph neural network (GNN) for extracting structural embeddings from CO instances with an LLM conditioned on these embeddings to identify high-performing algorithms in the form of solver-specific codes. This composite architecture ensures syntactic correctness, preserves problem topology, and aligns with natural language objectives, while an evolutionary refinement process iteratively optimizes generated algorithm. Extensive evaluations across Mixed Integer Linear Programming and Boolean Satisfiability problems, using nine benchmark datasets, demonstrate that our proposed STRCMP outperforms five strong neural and LLM-based methods by a large margin, in terms of both solution optimality and computational efficiency. The code and learned model will be publicly available upon the acceptance of the paper.