Learning to Imagine Manipulation Goals for Robot Task Planning

TL;DR

This paper introduces a neural network-based method for robot task planning that predicts multiple future outcomes, enabling robots to generate comprehensible, long-term plans in complex environments by combining machine learning and task planning techniques.

Contribution

It proposes a novel neural network model that encodes likely future outcomes of high-level actions, enhancing robot planning with long-term environment predictions.

Findings

Successfully applied to stacking and navigation tasks

Generated realistic multi-step environment predictions

Enabled robots to plan with foresight in cluttered environments

Abstract

Prospection is an important part of how humans come up with new task plans, but has not been explored in depth in robotics. Predicting multiple task-level is a challenging problem that involves capturing both task semantics and continuous variability over the state of the world. Ideally, we would combine the ability of machine learning to leverage big data for learning the semantics of a task, while using techniques from task planning to reliably generalize to new environment. In this work, we propose a method for learning a model encoding just such a representation for task planning. We learn a neural net that encodes the $k$ most likely outcomes from high level actions from a given world. Our approach creates comprehensible task plans that allow us to predict changes to the environment many time steps into the future. We demonstrate this approach via application to a stacking task in a cluttered environment, where the robot must select between different colored blocks while avoiding obstacles, in order to perform a task. We also show results on a simple navigation task. Our algorithm generates realistic image and pose predictions at multiple points in a given task.