Metaheuristics and Large Language Models Join Forces: Toward an Integrated Optimization Approach

TL;DR

This paper presents a hybrid approach integrating Large Language Models with metaheuristics to enhance solution quality in combinatorial optimization problems, demonstrating improved results and discussing limitations.

Contribution

It introduces a novel method using LLMs as pattern recognition tools within metaheuristics, advancing the integration of AI and optimization techniques.

Findings

LLMs improve metaheuristic solution quality.

The hybrid approach outperforms existing methods.

Prompt design is crucial for effective knowledge extraction.

Abstract

Since the rise of Large Language Models (LLMs) a couple of years ago, researchers in metaheuristics (MHs) have wondered how to use their power in a beneficial way within their algorithms. This paper introduces a novel approach that leverages LLMs as pattern recognition tools to improve MHs. The resulting hybrid method, tested in the context of a social network-based combinatorial optimization problem, outperforms existing state-of-the-art approaches that combine machine learning with MHs regarding the obtained solution quality. By carefully designing prompts, we demonstrate that the output obtained from LLMs can be used as problem knowledge, leading to improved results. Lastly, we acknowledge LLMs' potential drawbacks and limitations and consider it essential to examine them to advance this type of research further. Our method can be reproduced using a tool available at: https://github.com/camilochs/optipattern.