An In-depth Study of LLM Contributions to the Bin Packing Problem

TL;DR

This paper critically evaluates the role of Large Language Models in generating heuristics for the bin packing problem, revealing their limitations and proposing simpler, more effective algorithms based on detailed analysis.

Contribution

It provides a thorough analysis of LLM-generated heuristics for bin packing, highlights their opaqueness, and introduces new, interpretable algorithms tailored to specific instances.

Findings

LLM heuristics are largely opaque to experts.

Proposed algorithms are simpler, more efficient, and more interpretable.

The bin packing instances studied are relatively simple.

Abstract

Recent studies have suggested that Large Language Models (LLMs) could provide interesting ideas contributing to mathematical discovery. This claim was motivated by reports that LLM-based genetic algorithms produced heuristics offering new insights into the online bin packing problem under uniform and Weibull distributions. In this work, we reassess this claim through a detailed analysis of the heuristics produced by LLMs, examining both their behavior and interpretability. Despite being human-readable, these heuristics remain largely opaque even to domain experts. Building on this analysis, we propose a new class of algorithms tailored to these specific bin packing instances. The derived algorithms are significantly simpler, more efficient, more interpretable, and more generalizable, suggesting that the considered instances are themselves relatively simple. We then discuss the limitations of the claim regarding LLMs' contribution to this problem, which appears to rest on the mistaken assumption that the instances had previously been studied. Our findings instead emphasize the need for rigorous validation and contextualization when assessing the scientific value of LLM-generated outputs.