PerFACT: Motion Policy with LLM-Powered Dataset Synthesis and Fusion Action-Chunking Transformers

TL;DR

This paper introduces PerFACT, a novel motion planning framework that uses large language models for dataset synthesis and a fusion action-chunking transformer to improve generalization and speed in robotic motion planning.

Contribution

The paper presents a new large-scale dataset generation method with LLMs and a fusion transformer-based neural motion planner, enhancing generalization and efficiency over existing methods.

Findings

MpiNetsFusion plans several times faster than state-of-the-art methods.

Collected 3.5 million trajectories for training and evaluation.

The approach improves generalization to out-of-distribution scenarios.

Abstract

Deep learning methods have significantly enhanced motion planning for robotic manipulators by leveraging prior experiences within planning datasets. However, state-of-the-art neural motion planners are primarily trained on small datasets collected in manually generated workspaces, limiting their generalizability to out-of-distribution scenarios. Additionally, these planners often rely on monolithic network architectures that struggle to encode critical planning information. To address these challenges, we introduce Motion Policy with Dataset Synthesis powered by large language models (LLMs) and Fusion Action-Chunking Transformers (PerFACT), which incorporates two key components. Firstly, a novel LLM-powered workspace generation method, MotionGeneralizer, enables large-scale planning data collection by producing a diverse set of semantically feasible workspaces. Secondly, we introduce Fusion Motion Policy Networks (MpiNetsFusion), a generalist neural motion planner that uses a fusion action-chunking transformer to better encode planning signals and attend to multiple feature modalities. Leveraging MotionGeneralizer, we collect 3.5M trajectories to train and evaluate MpiNetsFusion against state-of-the-art planners, which shows that the proposed MpiNetsFusion can plan several times faster on the evaluated tasks.