The Art of Being Difficult: Combining Human and AI Strengths to Find Adversarial Instances for Heuristics

TL;DR

This paper showcases how combining human expertise with LLMs can generate adversarial instances in combinatorial optimization, leading to breakthroughs in longstanding heuristic bounds.

Contribution

It introduces a collaborative approach that refines LLM-generated outputs with human insight to improve lower bounds for heuristics in key combinatorial problems.

Findings

Achieved state-of-the-art lower bounds for heuristics in multiple problems.

Demonstrated the effectiveness of human-LLM collaboration in mathematical research.

Identified improved constructions for problems unsolved for over a decade.

Abstract

We demonstrate the power of human-LLM collaboration in tackling open problems in theoretical computer science. Focusing on combinatorial optimization, we refine outputs from the FunSearch algorithm [Romera-Paredes et al., Nature 2023] to derive state-of-the-art lower bounds for standard heuristics. Specifically, we target the generation of adversarial instances where these heuristics perform poorly. By iterating on FunSearch's outputs, we identify improved constructions for hierarchical $k$-median clustering, bin packing, the knapsack problem, and a generalization of Lov\'asz's gasoline problem - some of these have not seen much improvement for over a decade, despite intermittent attention. These results illustrate how expert oversight can effectively extrapolate algorithmic insights from LLM-based evolutionary methods to break long-standing barriers. Our findings demonstrate that while LLMs provide critical initial patterns, human expertise is essential for transforming these patterns into mathematically rigorous and insightful constructions. This work highlights that LLMs are a strong collaborative tool in mathematics and computer science research.