Self-Guiding Exploration for Combinatorial Problems

TL;DR

This paper introduces Self-Guiding Exploration, a novel prompting strategy for Large Language Models, significantly improving their ability to solve complex combinatorial problems and outperforming existing methods in optimization and reasoning tasks.

Contribution

The paper presents the first application of LLMs to a broad range of combinatorial problems using Self-Guiding Exploration, enhancing performance over existing prompting strategies.

Findings

SGE outperforms existing prompting strategies by over 27.84% in CP optimization.

SGE achieves 2.46% higher accuracy in other reasoning tasks.

First demonstration of LLMs effectively addressing diverse combinatorial problems.

Abstract

Large Language Models (LLMs) have become pivotal in addressing reasoning tasks across diverse domains, including arithmetic, commonsense, and symbolic reasoning. They utilize prompting techniques such as Exploration-of-Thought, Decomposition, and Refinement to effectively navigate and solve intricate tasks. Despite these advancements, the application of LLMs to Combinatorial Problems (CPs), known for their NP-hardness and critical roles in logistics and resource management remains underexplored. To address this gap, we introduce a novel prompting strategy: Self-Guiding Exploration (SGE), designed to enhance the performance of solving CPs. SGE operates autonomously, generating multiple thought trajectories for each CP task. It then breaks these trajectories down into actionable subtasks, executes them sequentially, and refines the results to ensure optimal outcomes. We present our research as the first to apply LLMs to a broad range of CPs and demonstrate that SGE outperforms existing prompting strategies by over 27.84% in CP optimization performance. Additionally, SGE achieves a 2.46% higher accuracy over the best existing results in other reasoning tasks (arithmetic, commonsense, and symbolic).