Generalizable Task Planning through Representation Pretraining

TL;DR

This paper introduces a method for multi-step manipulation planning that leverages object-level representations from synthetic datasets, enabling robots to generalize to new objects and outperform existing end-to-end approaches.

Contribution

It presents a novel learning-to-plan approach using scene understanding data to improve generalization in manipulation tasks for robots.

Findings

Achieves higher success rates than state-of-the-art methods.

Demonstrates effective generalization to unseen object instances.

Validates approach on household-inspired manipulation tasks.

Abstract

The ability to plan for multi-step manipulation tasks in unseen situations is crucial for future home robots. But collecting sufficient experience data for end-to-end learning is often infeasible in the real world, as deploying robots in many environments can be prohibitively expensive. On the other hand, large-scale scene understanding datasets contain diverse and rich semantic and geometric information. But how to leverage such information for manipulation remains an open problem. In this paper, we propose a learning-to-plan method that can generalize to new object instances by leveraging object-level representations extracted from a synthetic scene understanding dataset. We evaluate our method with a suite of challenging multi-step manipulation tasks inspired by household activities and show that our model achieves measurably better success rate than state-of-the-art end-to-end approaches. Additional information can be found at https://sites.google.com/view/gentp