ACCORD: Autoregressive Constraint-satisfying Generation for COmbinatorial Optimization with Routing and Dynamic attention
Henrik Abgaryan, Tristan Cazenave, Ararat Harutyunyan
TL;DR
ACCORD introduces a novel LLM-based framework that dynamically enforces feasibility constraints and activates problem-specific modules to effectively solve a wide range of NP-hard combinatorial optimization problems.
Contribution
This work presents the first large-scale, end-to-end LLM framework for multiple NP-hard combinatorial problems, leveraging dynamic constraint enforcement and attention-based routing.
Findings
ACCORD outperforms standard prompting and larger models like GPT-4.
The output structure improves solution feasibility.
The framework is effective across six diverse NP-hard problems.
Abstract
Large Language Models (LLMs) have demonstrated impressive reasoning capabilities, yet their direct application to NP-hard combinatorial problems (CPs) remains underexplored. In this work, we systematically investigate the reasoning abilities of LLMs on a variety of NP-hard combinatorial optimization tasks and introduce ACCORD: Autoregressive Constraint-satisfying generation for COmbinatorial optimization with Routing and Dynamic attention. ACCORD features a novel dataset representation and model architecture that leverage the autoregressive nature of LLMs to dynamically enforce feasibility constraints, coupled with attention-based routing to activate problem-specific LoRA modules. We also present the ACCORD-90k supervised dataset, covering six NP-hard combinatorial problems: TSP, VRP, Knapsack, FlowShop, JSSP, and BinPacking. Extensive experiments demonstrate that our ACCORD model,…
Peer Reviews
Decision·ICLR 2026 Conference Desk Rejected Submission
1. The idea of fine-tuning an LLM for CO is interesting. 2. Generating solutions under an autoregressive decoding framework ensures that feasibility constraints can be enforced step-by-step during generation. 3. The authors have provided the source code.
1. The presentation needs further improvement. For example: - In line 67, the capitalized "*A*ttention based *D*ynamic router" and the appearance of "5" are confusing. - Figure 1 does not illustrate the autoregressive decoding process of solution generation. After reading the figure and its caption, it initially seemed that the model samples solutions from an output probability distribution over nodes/tokens (e.g., via a heatmap), and only checks feasibility after generating the entire sol
* The empirical results suggest that the suggested format with step-by-step feasibility checks improves LLM performance on CO tasks. * The evaluation covers six different NP-hard tasks. * The paper additionally contributes the ACCORD-90K dataset as a new benchmark.
1. The dynamic routing adds significant complexity but seems to offer limited utility. In practice one could simply select the LoRA based on the given CO task or train one bigger combined LoRA for all given CO tasks. 2. It is not clear that a LoRA is needed for an LLM to follow the suggested output format. Instead, the output format may be acquired through in-context learning. The system prompt could specify the ACCORD format for the given CO problem and provide few-shot examples for some proble
* Fine-tuning LLMs to directly solve CO problems is an interesting yet challenging direction. * The authors provide source code to support reproducibility (but please use an anonymous link).
* The writing quality of this paper should be improved. For instance, some citation format should follow the correct convention (e.g., use `\citep`). * As acknowledged by the authors in lines 75–78, LLMs are originally designed for natural language generation rather than NP-hard optimization problems, which require complex search within constrained spaces. Given this, why do the authors choose to fine-tune such LLMs for directly solving CO problems? Would it not be more suitable to train a small
Using LLMs to solve combinatorial optimization problems is a hot topic and will likely remain an active area of research in the future.
The paper contains numerous confusing claims and several serious errors, it must to be written more carefully. (Please refer to the questions section for details.)
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsConstraint Satisfaction and Optimization