DHEvo: Data-Algorithm Based Heuristic Evolution for Generalizable MILP Solving

TL;DR

DHEvo introduces a data-algorithm co-evolution framework leveraging LLMs to generate and refine primal heuristics tailored for specific MILP problem classes, significantly improving generalization and solving efficiency.

Contribution

The paper proposes a novel co-evolution approach that combines data selection and heuristic evolution using LLMs, addressing generalization issues in MILP solving.

Findings

Outperforms existing heuristics and LLM-based methods on diverse benchmarks.

Effectively captures problem class characteristics for better heuristic generalization.

Significantly improves MILP solving efficiency across various problem instances.

Abstract

Primal heuristics play a critical role in improving the efficiency of mixed integer programming (MILP) solvers. As large language models (LLMs) have demonstrated superior code generation abilities, recent MILP works are devoted to leveraging the evolutionary computation approaches with LLMs to generate effective primal heuristics. Although the generated heuristics have achieved better solving performance than the hand-crafted ones with little adaptability, the advantage of current LLM-based methods is limited to few MILP instances in one problem class, as they fail to capture the instance characteristics in the problem class (the MILP instances generated from the same mathematical model are defined as a problem class). Since MILP instances often differ significantly in structure and feature distribution, the neglect of their characteristics in the evolution process results in poor generalization within the same problem class. To overcome this challenge, we propose a data-algorithm co-evolution framework (DHEvo) that iteratively selects representative instances and evolves corresponding heuristics. With the initial instance distribution, we develop an LLM-based multi-agent system to generate data-code pairs simultaneously. These data-code pairs are iteratively refined based on their fitness scores, leading to the identification of the most effective heuristic over the entire problem class. Extensive experiments across diverse MILP benchmarks demonstrate that our approach significantly outperforms both human-designed heuristics and existing LLM-based methods.