Combined Task and Action Learning from Human Demonstrations for Mobile Manipulation Applications

TL;DR

This paper introduces a probabilistic framework for mobile manipulation learning from human demonstrations, enabling robots to infer task goals and generate feasible actions in new scenarios without explicit goal specifications.

Contribution

It presents a novel approach combining task goal representation and flexible action modeling using Monte Carlo tree search, improving generalization from limited demonstrations.

Findings

Effective in complex real-world tasks

Enables goal inference without explicit specifications

Generates feasible trajectories considering multiple outcomes

Abstract

Learning from demonstrations is a promising paradigm for transferring knowledge to robots. However, learning mobile manipulation tasks directly from a human teacher is a complex problem as it requires learning models of both the overall task goal and of the underlying actions. Additionally, learning from a small number of demonstrations often introduces ambiguity with respect to the intention of the teacher, making it challenging to commit to one model for generalizing the task to new settings. In this paper, we present an approach to learning flexible mobile manipulation action models and task goal representations from teacher demonstrations. Our action models enable the robot to consider different likely outcomes of each action and to generate feasible trajectories for achieving them. Accordingly, we leverage a probabilistic framework based on Monte Carlo tree search to compute sequences of feasible actions imitating the teacher intention in new settings without requiring the teacher to specify an explicit goal state. We demonstrate the effectiveness of our approach in complex tasks carried out in real-world settings.