Creating and Repairing Robot Programs in Open-World Domains

TL;DR

This paper introduces RoboRepair, a system that uses LLMs to generate recovery programs for faulty robot tasks, enabling effective error recovery in open-world domains.

Contribution

The paper presents RoboRepair, a novel approach that traces execution errors and generates recovery programs to improve robot task robustness using LLMs.

Findings

RoboRepair effectively recovers from errors in diverse tasks.

The system outperforms baseline methods in recovery efficiency.

A new benchmark with eleven tasks evaluates recovery performance.

Abstract

Using Large Language Models (LLMs) to produce robot programs from natural language has allowed for robot systems that can complete a higher diversity of tasks. However, LLM-generated programs may be faulty, either due to ambiguity in instructions, misinterpretation of the desired task, or missing information about the world state. As these programs run, the state of the world changes and they gather new information. When a failure occurs, it is important that they recover from the current world state and avoid repeating steps that they they previously completed successfully. We propose RoboRepair, a system which traces the execution of a program up until error, and then runs an LLM-produced recovery program that minimizes repeated actions. To evaluate the efficacy of our system, we create a benchmark consisting of eleven tasks with various error conditions that require the generation of a recovery program. We compare the efficiency of the recovery program to a plan built with an oracle that has foreknowledge of future errors.