RoboCritics: Enabling Reliable End-to-End LLM Robot Programming through Expert-Informed Critics

TL;DR

RoboCritics enhances LLM-based robot programming by integrating expert-informed critics that analyze motion safety, providing transparent feedback and iterative refinement to improve reliability and safety in end-user robot programming.

Contribution

This work introduces RoboCritics, a novel framework combining LLMs with motion-level critics to improve safety and transparency in robot programming.

Findings

Reduced safety violations in robot execution

Improved user verification and refinement of programs

Enhanced reliability and safety in end-user robot programming

Abstract

End-user robot programming grants users the flexibility to re-task robots in situ, yet it remains challenging for novices due to the need for specialized robotics knowledge. Large Language Models (LLMs) hold the potential to lower the barrier to robot programming by enabling task specification through natural language. However, current LLM-based approaches generate opaque, "black-box" code that is difficult to verify or debug, creating tangible safety and reliability risks in physical systems. We present RoboCritics, an approach that augments LLM-based robot programming with expert-informed motion-level critics. These critics encode robotics expertise to analyze motion-level execution traces for issues such as joint speed violations, collisions, and unsafe end-effector poses. When violations are detected, critics surface transparent feedback and offer one-click fixes that forward structured messages back to the LLM, enabling iterative refinement while keeping users in the loop. We instantiated RoboCritics in a web-based interface connected to a UR3e robot and evaluated it in a between-subjects user study (n=18). Compared to a baseline LLM interface, RoboCritics reduced safety violations, improved execution quality, and shaped how participants verified and refined their programs. Our findings demonstrate that RoboCritics enables more reliable and user-centered end-to-end robot programming with LLMs.