PathWise: Planning through World Model for Automated Heuristic Design via Self-Evolving LLMs

TL;DR

PathWise introduces a multi-agent framework that leverages a world model and reasoning to improve automated heuristic design for combinatorial optimization, enabling faster convergence and better generalization.

Contribution

It presents a novel planning-based approach using self-evolving LLMs with a world model, moving beyond trial-and-error to state-aware heuristic generation.

Findings

Faster convergence to high-quality heuristics

Generalizes across different LLMs and problem sizes

Scales effectively to larger combinatorial problems

Abstract

Large Language Models (LLMs) have enabled automated heuristic design (AHD) for combinatorial optimization problems (COPs), but existing frameworks' reliance on fixed evolutionary rules and static prompt templates often leads to myopic heuristic generation, redundant evaluations, and limited reasoning about how new heuristics should be derived. We propose a novel multi-agent reasoning framework, referred to as Planning through World Model for Automated Heuristic Design via Self-Evolving LLMs (PathWise), which formulates heuristic generation as a sequential decision process over an entailment graph serving as a compact, stateful memory of the search trajectory. This approach allows the system to carry forward past decisions and reuse or avoid derivation information across generations. A policy agent plans evolutionary actions, a world model agent generates heuristic rollouts conditioned on those actions, and critic agents provide routed reflections summarizing lessons from prior steps, shifting LLM-based AHD from trial-and-error evolution toward state-aware planning through reasoning. Experiments across diverse COPs show that PathWise converges faster to better heuristics, generalizes across different LLM backbones, and scales to larger problem sizes.