Transformers are Adaptable Task Planners

TL;DR

This paper introduces a Transformer-based task planner that learns from demonstrations, generalizes to new preferences with minimal data, and is validated both in simulation and real-world robotic dish rearrangement tasks.

Contribution

The paper presents a novel Transformer Task Planner (TTP) that leverages object attributes to learn high-level actions and generalizes to unseen preferences using minimal demonstrations.

Findings

TTP can be pre-trained on multiple preferences.

TTP generalizes to unseen preferences with a single demonstration.

Successful real-world dish rearrangement with a Franka Panda robot.

Abstract

Every home is different, and every person likes things done in their particular way. Therefore, home robots of the future need to both reason about the sequential nature of day-to-day tasks and generalize to user's preferences. To this end, we propose a Transformer Task Planner(TTP) that learns high-level actions from demonstrations by leveraging object attribute-based representations. TTP can be pre-trained on multiple preferences and shows generalization to unseen preferences using a single demonstration as a prompt in a simulated dishwasher loading task. Further, we demonstrate real-world dish rearrangement using TTP with a Franka Panda robotic arm, prompted using a single human demonstration.