CALM: Co-evolution of Algorithms and Language Model for Automatic Heuristic Design

TL;DR

This paper introduces CALM, a hybrid framework that co-evolves language models and algorithms for automatic heuristic design, significantly improving optimization performance with minimal computational resources.

Contribution

It presents a novel hybrid approach combining verbal and numerical guidance through reinforcement learning to co-evolve LLMs and heuristics, outperforming existing methods.

Findings

Outperforms state-of-the-art baselines across various tasks.

Operates efficiently on a single 24GB GPU with a 7B model.

Surpasses methods relying solely on verbal guidance.

Abstract

Tackling complex optimization problems often relies on expert-designed heuristics, typically crafted through extensive trial and error. Recent advances demonstrate that large language models (LLMs), when integrated into well-designed evolutionary search frameworks, can autonomously discover high-performing heuristics at a fraction of the traditional cost. However, existing approaches predominantly rely on verbal guidance, i.e., manipulating the prompt generation process, to steer the evolution of heuristics, without adapting the underlying LLM. We propose a hybrid framework that combines verbal and numerical guidance, the latter achieved by fine-tuning the LLM via reinforcement learning based on the quality of generated heuristics. This joint optimization allows the LLM to co-evolve with the search process. Our method outperforms state-of-the-art (SOTA) baselines across various optimization tasks, running locally on a single 24GB GPU using a 7B model with INT4 quantization. It surpasses methods that rely solely on verbal guidance, even when those use significantly more powerful API-based models.