Interactive Disambiguation for Behavior Tree Execution

TL;DR

This paper presents an interactive system that learns behavior trees from demonstration and uses verbal clarification to resolve ambiguities during robot pick-and-place tasks in unpredictable environments.

Contribution

It introduces a novel integrated approach combining behavior tree learning from demonstration with verbal interaction for disambiguation during execution.

Findings

System successfully disambiguates objects in complex scenarios.

Demonstrates effective learning of behavior trees from demonstrations.

Code is publicly available for further research.

Abstract

In recent years, robots are used in an increasing variety of tasks, especially by small- and medium- sized enterprises. These tasks are usually fast-changing, they have a collaborative scenario and happen in unpredictable environments with possible ambiguities. It is important to have methods capable of generating robot programs easily, that are made as general as possible by handling uncertainties. We present a system that integrates a method to learn Behavior Trees (BTs) from demonstration for pick and place tasks, with a framework that uses verbal interaction to ask follow-up clarification questions to resolve ambiguities. During the execution of a task, the system asks for user input when there is need to disambiguate an object in the scene, when the targets of the task are objects of a same type that are present in multiple instances. The integrated system is demonstrated on different scenarios of a pick and place task, with increasing level of ambiguities. The code used for this paper is made publicly available.