VerifyLLM: LLM-Based Pre-Execution Task Plan Verification for Robots

TL;DR

This paper introduces VerifyLLM, an architecture that uses Large Language Models to automatically verify high-level robotic task plans before execution, enhancing reliability and reducing errors.

Contribution

It presents a novel method combining LLMs and LTL conversion for pre-execution plan verification in robotics, which is a new approach in this domain.

Findings

Effective verification of task plans in simulated and real environments.

Broad applicability to household robotic tasks.

Improved reliability of robotic task planning.

Abstract

In the field of robotics, researchers face a critical challenge in ensuring reliable and efficient task planning. Verifying high-level task plans before execution significantly reduces errors and enhance the overall performance of these systems. In this paper, we propose an architecture for automatically verifying high-level task plans before their execution in simulator or real-world environments. Leveraging Large Language Models (LLMs), our approach consists of two key steps: first, the conversion of natural language instructions into Linear Temporal Logic (LTL), followed by a comprehensive analysis of action sequences. The module uses the reasoning capabilities of the LLM to evaluate logical coherence and identify potential gaps in the plan. Rigorous testing on datasets of varying complexity demonstrates the broad applicability of the module to household tasks. We contribute to improving the reliability and efficiency of task planning and addresses the critical need for robust pre-execution verification in autonomous systems. The code is available at https://verifyllm.github.io.