Deep Insights into Automated Optimization with Large Language Models and Evolutionary Algorithms

TL;DR

This paper reviews and proposes a new paradigm combining Large Language Models and Evolutionary Algorithms to automate and enhance optimization processes across diverse problem domains.

Contribution

It introduces a novel LLM-EA paradigm for automated optimization and provides an in-depth analysis of its key components and challenges.

Findings

Comprehensive review of LLMs in optimization

Proposal of a new LLM-EA paradigm

Analysis of key components and challenges

Abstract

Designing optimization approaches, whether heuristic or meta-heuristic, usually demands extensive manual intervention and has difficulty generalizing across diverse problem domains. The combination of Large Language Models (LLMs) and Evolutionary Algorithms (EAs) offers a promising new approach to overcome these limitations and make optimization more automated. In this setup, LLMs act as dynamic agents that can generate, refine, and interpret optimization strategies, while EAs efficiently explore complex solution spaces through evolutionary operators. Since this synergy enables a more efficient and creative search process, we first conduct an extensive review of recent research on the application of LLMs in optimization. We focus on LLMs' dual functionality as solution generators and algorithm designers. Then, we summarize the common and valuable designs in existing work and propose a novel LLM-EA paradigm for automated optimization. Furthermore, centered on this paradigm, we conduct an in-depth analysis of innovative methods for three key components: individual representation, variation operators, and fitness evaluation. We address challenges related to heuristic generation and solution exploration, especially from the LLM prompts' perspective. Our systematic review and thorough analysis of the paradigm can assist researchers in better understanding the current research and promoting the development of combining LLMs with EAs for automated optimization.