Cocobo: Exploring Large Language Models as the Engine for End-User Robot Programming

TL;DR

Cocobo leverages large language models to enable end-users with no coding experience to program robots through natural language and interactive diagrams, simplifying robot customization.

Contribution

This paper introduces Cocobo, a novel natural language programming system that uses LLMs for understanding, generating, and explaining robot programs with interactive diagrams.

Findings

Users with zero coding experience can successfully customize robot programs.

Cocobo has a low learning curve for end-users.

The system effectively translates natural language into executable robot code.

Abstract

End-user development allows everyday users to tailor service robots or applications to their needs. One user-friendly approach is natural language programming. However, it encounters challenges such as an expansive user expression space and limited support for debugging and editing, which restrict its application in end-user programming. The emergence of large language models (LLMs) offers promising avenues for the translation and interpretation between human language instructions and the code executed by robots, but their application in end-user programming systems requires further study. We introduce Cocobo, a natural language programming system with interactive diagrams powered by LLMs. Cocobo employs LLMs to understand users' authoring intentions, generate and explain robot programs, and facilitate the conversion between executable code and flowchart representations. Our user study shows that Cocobo has a low learning curve, enabling even users with zero coding experience to customize robot programs successfully.