irace-evo: Automatic Algorithm Configuration Extended With LLM-Based Code Evolution

TL;DR

irace-evo extends automatic algorithm configuration by integrating large language model-driven code evolution, enabling joint exploration of parameters and code, leading to improved algorithms at low cost.

Contribution

This paper introduces irace-evo, combining parameter tuning with LLM-based code evolution, supporting multiple languages and ensuring robust code modifications.

Findings

irace-evo discovers superior algorithm variants for VSBPP.

It achieves competitive improvements with lightweight models at low cost.

The framework effectively combines configuration and code evolution for heuristic optimization.

Abstract

Automatic algorithm configuration tools such as irace efficiently tune parameter values but leave algorithmic code unchanged. This paper introduces a first version of irace-evo, an extension of irace that integrates code evolution through large language models (LLMs) to jointly explore parameter and code spaces. The proposed framework enables multi-language support (e.g., C++, Python), reduces token consumption via progressive context management, and employs the Always-From-Original principle to ensure robust and controlled code evolution. We evaluate irace-evo on the Construct, Merge, Solve & Adapt (CMSA) metaheuristic for the Variable-Sized Bin Packing Problem (VSBPP). Experimental results show that irace-evo can discover new algorithm variants that outperform the state-of-the-art CMSA implementation while maintaining low computational and monetary costs. Notably, irace-evo generates competitive algorithmic improvements using lightweight models (e.g., Claude Haiku 3.5) with a total usage cost under 2 euros. These results demonstrate that coupling automatic configuration with LLM-driven code evolution provides a powerful, cost-efficient avenue for advancing heuristic design and metaheuristic optimization.