Transformers are Adaptable Task Planners
Vidhi Jain, Yixin Lin, Eric Undersander, Yonatan Bisk, Akshara Rai

TL;DR
This paper introduces a Transformer-based task planner that learns from demonstrations, generalizes to new preferences with minimal data, and is validated both in simulation and real-world robotic dish rearrangement tasks.
Contribution
The paper presents a novel Transformer Task Planner (TTP) that leverages object attributes to learn high-level actions and generalizes to unseen preferences using minimal demonstrations.
Findings
TTP can be pre-trained on multiple preferences.
TTP generalizes to unseen preferences with a single demonstration.
Successful real-world dish rearrangement with a Franka Panda robot.
Abstract
Every home is different, and every person likes things done in their particular way. Therefore, home robots of the future need to both reason about the sequential nature of day-to-day tasks and generalize to user's preferences. To this end, we propose a Transformer Task Planner(TTP) that learns high-level actions from demonstrations by leveraging object attribute-based representations. TTP can be pre-trained on multiple preferences and shows generalization to unseen preferences using a single demonstration as a prompt in a simulated dishwasher loading task. Further, we demonstrate real-world dish rearrangement using TTP with a Franka Panda robotic arm, prompted using a single human demonstration.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsReinforcement Learning in Robotics · Robot Manipulation and Learning · Human Pose and Action Recognition
MethodsAttention Is All You Need · Linear Layer · Softmax · Multi-Head Attention · Residual Connection · Dense Connections · Position-Wise Feed-Forward Layer · Adam · Dropout · Label Smoothing
