Transferable Task Execution from Pixels through Deep Planning Domain Learning

TL;DR

This paper introduces Deep Planning Domain Learning (DPDL), a hierarchical approach that combines visual learning and symbolic planning to enable robots to generalize manipulation tasks in complex, real-world environments.

Contribution

The paper presents DPDL, a novel method that integrates deep visual models with symbolic planning for transferable robotic task execution.

Findings

DPDL enables robots to perform multi-step manipulation tasks in photorealistic environments.

The hierarchical model improves generalization to new tasks without explicit training.

Combines strengths of deep learning and symbolic planning for real-world robotics.

Abstract

While robots can learn models to solve many manipulation tasks from raw visual input, they cannot usually use these models to solve new problems. On the other hand, symbolic planning methods such as STRIPS have long been able to solve new problems given only a domain definition and a symbolic goal, but these approaches often struggle on the real world robotic tasks due to the challenges of grounding these symbols from sensor data in a partially-observable world. We propose Deep Planning Domain Learning (DPDL), an approach that combines the strengths of both methods to learn a hierarchical model. DPDL learns a high-level model which predicts values for a large set of logical predicates consisting of the current symbolic world state, and separately learns a low-level policy which translates symbolic operators into executable actions on the robot. This allows us to perform complex, multi-step tasks even when the robot has not been explicitly trained on them. We show our method on manipulation tasks in a photorealistic kitchen scenario.