Work in Progress -- Automated Generation of Robotic Planning Domains from Observations

TL;DR

This paper presents a system that automatically generates robotic planning operators from human demonstrations, enabling robots to learn and plan actions based on observed behaviors, with deployment on a simulated TIAGo robot.

Contribution

The work introduces an automatic method for extracting planning operators from demonstrations, integrating segmentation, recognition, and symbolic planning for robotic applications.

Findings

Successfully generated planning operators from human demonstrations

Deployed generated plans on a simulated TIAGo robot

Outlined future research directions for learning from failures

Abstract

In this paper, we report the results of our latest work on the automated generation of planning operators from human demonstrations, and we present some of our future research ideas. To automatically generate planning operators, our system segments and recognizes different observed actions from human demonstrations. We then proposed an automatic extraction method to detect the relevant preconditions and effects from these demonstrations. Finally, our system generates the associated planning operators and finds a sequence of actions that satisfies a user-defined goal using a symbolic planner. The plan is deployed on a simulated TIAGo robot. Our future research directions include learning from and explaining execution failures and detecting cause-effect relationships between demonstrated hand activities and their consequences on the robot's environment. The former is crucial for trust-based and efficient human-robot collaboration and the latter for learning in realistic and dynamic environments.