Discovering Heuristics with Large Language Models (LLMs) for Mixed-Integer Programs: Single-Machine Scheduling

TL;DR

This paper demonstrates that Large Language Models can discover effective heuristics for the single-machine total tardiness scheduling problem, outperforming traditional methods especially on large, complex instances, showcasing human-LLM collaboration's potential in combinatorial optimization.

Contribution

The study introduces two novel LLM-discovered heuristics, EDDC and MDDC, for the SMTT problem, advancing the use of LLMs in generating scalable, high-performing optimization heuristics.

Findings

EDD C improves upon classic EDD rule on large instances

MDD C outperforms traditional heuristics and rivals exact methods

LLM-discovered heuristics are effective for NP-hard scheduling problems

Abstract

Our study contributes to the scheduling and combinatorial optimization literature with new heuristics discovered by leveraging the power of Large Language Models (LLMs). We focus on the single-machine total tardiness (SMTT) problem, which aims to minimize total tardiness by sequencing n jobs on a single processor without preemption, given processing times and due dates. We develop and benchmark two novel LLM-discovered heuristics, the EDD Challenger (EDDC) and MDD Challenger (MDDC), inspired by the well-known Earliest Due Date (EDD) and Modified Due Date (MDD) rules. In contrast to prior studies that employed simpler rule-based heuristics, we evaluate our LLM-discovered algorithms using rigorous criteria, including optimality gaps and solution time derived from a mixed-integer programming (MIP) formulation of SMTT. We compare their performance against state-of-the-art heuristics and exact methods across various job sizes (20, 100, 200, and 500 jobs). For instances with more than 100 jobs, exact methods such as MIP and dynamic programming become computationally intractable. Up to 500 jobs, EDDC improves upon the classic EDD rule and another widely used algorithm in the literature. MDDC consistently outperforms traditional heuristics and remains competitive with exact approaches, particularly on larger and more complex instances. This study shows that human-LLM collaboration can produce scalable, high-performing heuristics for NP-hard constrained combinatorial optimization, even under limited resources when effectively configured.