Instructing Hierarchical Tasks to Robots by Verbal Commands

TL;DR

This paper presents a framework enabling robots to understand and execute hierarchical tasks based on verbal commands, facilitating natural language interaction and collaboration, with demonstrated effectiveness in diverse tasks.

Contribution

The work introduces an open-source framework that grounds verbal commands to physical actions and targets, supporting hierarchical task instructions for robots.

Findings

Effective verbal command grounding for robot actions

Supports hierarchical task instructions with natural language

Demonstrated in human-robot collaboration scenarios

Abstract

Natural language is an effective tool for communication, as information can be expressed in different ways and at different levels of complexity. Verbal commands, utilized for instructing robot tasks, can therefor replace traditional robot programming techniques, and provide a more expressive means to assign actions and enable collaboration. However, the challenge of utilizing speech for robot programming is how actions and targets can be grounded to physical entities in the world. In addition, to be time-efficient, a balance needs to be found between fine- and course-grained commands and natural language phrases. In this work we provide a framework for instructing tasks to robots by verbal commands. The framework includes functionalities for single commands to actions and targets, as well as longer-term sequences of actions, thereby providing a hierarchical structure to the robot tasks. Experimental evaluation demonstrates the functionalities of the framework by human collaboration with a robot in different tasks, with different levels of complexity. The tools are provided open-source at https://petim44.github.io/voice-jogger/