GOAL: A Generalist Combinatorial Optimization Agent Learner
Darko Drakulic, Sofia Michel, Jean-Marc Andreoli
TL;DR
GOAL is a versatile machine learning model designed to solve multiple combinatorial optimization problems efficiently, with the ability to adapt to new problems through fine-tuning, using a novel architecture that handles diverse graph-based data.
Contribution
The paper introduces GOAL, a unified model with a new mixed-attention architecture capable of solving various COPs and adapting to new problems with minimal retraining.
Findings
GOAL performs nearly as well as specialized models on multiple COPs.
It demonstrates strong transfer learning capabilities for new problems.
The architecture effectively handles heterogeneous graph data.
Abstract
Machine Learning-based heuristics have recently shown impressive performance in solving a variety of hard combinatorial optimization problems (COPs). However, they generally rely on a separate neural model, specialized and trained for each single problem. Any variation of a problem requires adjustment of its model and re-training from scratch. In this paper, we propose GOAL (for Generalist combinatorial Optimization Agent Learner), a generalist model capable of efficiently solving multiple COPs and which can be fine-tuned to solve new COPs. GOAL consists of a single backbone plus light-weight problem-specific adapters for input and output processing. The backbone is based on a new form of mixed-attention blocks which allows to handle problems defined on graphs with arbitrary combinations of node, edge and instance-level features. Additionally, problems which involve heterogeneous types…
Peer Reviews
Decision·ICLR 2025 Poster
（1）The paper is novel , as it designs a multi-task learning approach to solve various combinatorial optimization problems through an end-to-end model. The authors developed a mixed-attention block to effectively achieve this objective. （2）The paper is well-organized, concisely written, and has good readability. （3）This paper demonstrates substantial work, conducting experiments on various combinatorial optimization problems and showcasing the effectiveness of the proposed method in terms of so
（1）The description of dimension transformations and the learning process of the model is not very illustrative. It is recommended to add figure and text to enhance the explanation. （2）In Table 1, only one problem size is tested, and it is relatively small. It is recommended to include experiments with larger problem sizes. （3）The paper lacks a theoretical analysis of the method's effectiveness, and it is recommended to include this section. （4）There are few effective baselines in Table 1. For
* A generalized combinatorial optimization solver is favored by the research community. * The proposed GOAL transformer architecture seems interesting and promising, especially given the fact that it outperforms other neural networks when trained for a specific problem. * The generalized training and fine-tuning result seems sound and promising. The greedy version of GOAL is comparative to other greddy peer methods.
Some important details are missing in this paper: * What are the implementation details of the "codebook"? Please specify * Definitions of BQ-MDP and tail-recursive are needed in the main text to make this paper self-contained. Misc: * The first paragraph is too long * There are multiple misuses of \citep and \citet in this paper. For example, In L112, please use \citet for Khalil et al., 2017. In L116, please use \citep for Kool et al. (2019). Please proofread and fix all the misusages * If t
* The problem of multi-task pretraining on multiple CO problems is novel and interesting. * The main experiments are fairly comprehensive and consider a wide range of problems. * The reported results are promising and suggest that pretraining can significantly improve convergence speed when fine-tuning for a new problem.
I am skeptical about the architecture design for multi-type problems, which is why I currently rate this work as a borderline reject. If I understand correctly, two changes are made when working on a multi-type problem: 1. Multiple adaptors are learned, one for each node type (and edge type), respectively. 2. The same backbone model is applied separately to each type and each node type now applies the attention module twice: once to self-attend to other nodes of the same type and a second time t
Code & Models
Videos
Taxonomy
TopicsMetaheuristic Optimization Algorithms Research · Semantic Web and Ontologies
MethodsSoftmax · Attention Is All You Need · Sparse Evolutionary Training