Automating Manual Tasks through Intuitive Robot Programming and Cognitive Robotics

TL;DR

This paper introduces a new approach for intuitive robot programming using natural language and gestures, leveraging large language models and computer vision to enhance safety, transparency, and user acceptance.

Contribution

It presents a novel system that translates natural interactions into robot programs with feedback mechanisms for review and adjustment.

Findings

Effective translation of natural language and gestures into robot commands

Enhanced safety and transparency through review and adjustment

Improved user acceptance of robot programming methods

Abstract

This paper presents a novel concept for intuitive end-user programming of robots, inspired by natural interaction between humans. Natural language and supportive gestures are translated into robot programs using large language models (LLMs) and computer vision (CV). Through equally natural system feedback in the form of clarification questions and visual representations, the generated program can be reviewed and adjusted, thereby ensuring safety, transparency, and user acceptance.