Automating Manual Tasks through Intuitive Robot Programming and Cognitive Robotics
Bijan Kavousian, Petar Tesic, Oliver Petrovic, Christian Brecher

TL;DR
This paper introduces a new approach for intuitive robot programming using natural language and gestures, leveraging large language models and computer vision to enhance safety, transparency, and user acceptance.
Contribution
It presents a novel system that translates natural interactions into robot programs with feedback mechanisms for review and adjustment.
Findings
Effective translation of natural language and gestures into robot commands
Enhanced safety and transparency through review and adjustment
Improved user acceptance of robot programming methods
Abstract
This paper presents a novel concept for intuitive end-user programming of robots, inspired by natural interaction between humans. Natural language and supportive gestures are translated into robot programs using large language models (LLMs) and computer vision (CV). Through equally natural system feedback in the form of clarification questions and visual representations, the generated program can be reviewed and adjusted, thereby ensuring safety, transparency, and user acceptance.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
