Large Language Models as Common-Sense Heuristics

TL;DR

This paper introduces a novel planning method that uses Large Language Models as heuristics in Hill-Climbing Search, significantly improving success rates in household tasks without requiring intermediate language translation.

Contribution

The paper presents a new approach leveraging LLMs as heuristics for planning, avoiding intermediate language translation and enhancing success rates in household environments.

Findings

Outperforms similar systems by 22 percentage points in success rate

Generates consistently executable plans in original action representations

Eliminates the need for intermediate language translation

Abstract

While systems designed for solving planning tasks vastly outperform Large Language Models (LLMs) in this domain, they usually discard the rich semantic information embedded within task descriptions. In contrast, LLMs possess parametrised knowledge across a wide range of topics, enabling them to leverage the natural language descriptions of planning tasks in their solutions. However, current research in this direction faces challenges in generating correct and executable plans. Furthermore, these approaches depend on the LLM to output solutions in an intermediate language, which must be translated into the representation language of the planning task. We introduce a novel planning method, which leverages the parametrised knowledge of LLMs by using their output as a heuristic for Hill-Climbing Search. This approach is further enhanced by prompting the LLM to generate a solution estimate to guide the search. Our method outperforms the task success rate of similar systems within a common household environment by 22 percentage points, with consistently executable plans. All actions are encoded in their original representation, demonstrating that strong results can be achieved without an intermediate language, thus eliminating the need for a translation step.