Robo-Instruct: Simulator-Augmented Instruction Alignment For Finetuning Code LLMs

TL;DR

Robo-Instruct introduces a method to fine-tune small code LLMs for robot tasks by dynamically creating simulation environments and refining instructions, improving task-program alignment and performance.

Contribution

The paper presents Robo-Instruct, a novel approach that synthesizes simulation environments during execution and refines instructions, enabling effective fine-tuning of small code LLMs for robotics tasks.

Findings

Fine-tuned models outperform baseline methods.

Models match or surpass larger proprietary models.

Effective simulation environment synthesis during task execution.

Abstract

Code LLMs have shown promising results with converting tasks in natural language to programs that can be executed by service robots. We are interested in finetuning small, specialized LLMs for this purpose, but collecting datasets of task-program pairs specific to each robot is time-consuming and expensive. While approaches such as SELF-INSTRUCT and EVOL-INSTRUCT are capable of generating novel tasks given a few examples, they are unable to provide the corresponding programs that correctly abide by physical-world and robot-constraints using the provided programming interface. Using a simulator is a natural potential solution to checking for such constraints, but building simulation environments that can handle arbitrary tasks and their necessary objects and locations, is challenging. To address these challenges, we introduce ROBO-INSTRUCT, which synthesizes task-specific simulation environments on the fly during program execution, by opportunistically inferring entity properties and enforcing corresponding constraints based on how the entities are used in the task program. Additionally, ROBO-INSTRUCT integrates an LLM-aided post-processing procedure to refine instructions for better alignment with robot programs. We demonstrate the effectiveness of ROBO-INSTRUCT across multiple LLMs, showing that our fine-tuned models outperform all baseline methods and even match or surpass the performance of several larger and proprietary models.