TIDE: Tuning-Integrated Dynamic Evolution for LLM-Based Automated Heuristic Design

TL;DR

TIDE introduces a novel framework that combines structural diversity and parameter tuning in LLM-based heuristic design, leading to superior solutions and efficiency across various optimization problems.

Contribution

It presents a nested evolution framework with a Tree Similarity Edit Distance and UCB-based scheduling to improve heuristic discovery in LLM-driven optimization.

Findings

Outperforms state-of-the-art methods in solution quality.

Reduces computational costs and improves search efficiency.

Successfully applied to nine combinatorial optimization problems.

Abstract

Although Large Language Models have advanced Automated Heuristic Design, treating algorithm evolution as a monolithic text generation task overlooks the coupling between discrete algorithmic structures and continuous numerical parameters. Consequently, existing methods often discard promising algorithms due to uncalibrated constants and suffer from premature convergence resulting from simple similarity metrics. To address these limitations, we propose TIDE, a Tuning-Integrated Dynamic Evolution framework designed to decouple structural reasoning from parameter optimization. TIDE features a nested architecture where an outer parallel island model utilizes Tree Similarity Edit Distance to drive structural diversity, while an inner loop integrates LLM-based logic generation with a differential mutation operator for parameter tuning. Additionally, a UCB-based scheduler dynamically prioritizes high-yield prompt strategies to optimize resource allocation. Extensive experiments across nine combinatorial optimization problems demonstrate that TIDE discovers heuristics that significantly outperform state-of-the-art baselines in solution quality while achieving improved search efficiency and reduced computational costs.