Controlling the Mutation in Large Language Models for the Efficient Evolution of Algorithms

TL;DR

This paper presents a new method for controlling mutation in LLM-driven evolutionary algorithms, using dynamic prompts and power-law distributions to improve convergence and exploration in automated algorithm design.

Contribution

It introduces adaptive mutation prompts inspired by genetic algorithms, enabling better control over mutation rates in LLM-based evolutionary frameworks.

Findings

GPT-3.5-turbo struggles with mutation instructions

GPT-4o successfully adapts mutation rates based on prompts

Dynamic mutation rates enhance convergence speed and adaptability

Abstract

The integration of Large Language Models (LLMs) with evolutionary computation (EC) has introduced a promising paradigm for automating the design of metaheuristic algorithms. However, existing frameworks, such as the Large Language Model Evolutionary Algorithm (LLaMEA), often lack precise control over mutation mechanisms, leading to inefficiencies in solution space exploration and potentially suboptimal convergence. This paper introduces a novel approach to mutation control within LLM-driven evolutionary frameworks, inspired by theory of genetic algorithms. Specifically, we propose dynamic mutation prompts that adaptively regulate mutation rates, leveraging a heavy-tailed power-law distribution to balance exploration and exploitation. Experiments using GPT-3.5-turbo and GPT-4o models demonstrate that GPT-3.5-turbo fails to adhere to the specific mutation instructions, while GPT-4o is able to adapt its mutation based on the prompt engineered dynamic prompts. Further experiments show that the introduction of these dynamic rates can improve the convergence speed and adaptability of LLaMEA, when using GPT-4o. This work sets the starting point for better controlled LLM-based mutations in code optimization tasks, paving the way for further advancements in automated metaheuristic design.