Generalizable Long-Horizon Manipulations with Large Language Models

TL;DR

This paper presents a framework using Large Language Models to generate task conditions that enable robots to perform long-horizon manipulations with new objects and tasks, improving adaptability and generalization.

Contribution

The work introduces a novel LLM-based approach for generating task conditions to guide dynamic movement primitives in robotic manipulation, applicable to unseen objects and tasks.

Findings

Effective in simulated environments for long-horizon tasks

Successful transfer to real-world robotic manipulation

Enhances robot versatility with novel objects and tasks

Abstract

This work introduces a framework harnessing the capabilities of Large Language Models (LLMs) to generate primitive task conditions for generalizable long-horizon manipulations with novel objects and unseen tasks. These task conditions serve as guides for the generation and adjustment of Dynamic Movement Primitives (DMP) trajectories for long-horizon task execution. We further create a challenging robotic manipulation task suite based on Pybullet for long-horizon task evaluation. Extensive experiments in both simulated and real-world environments demonstrate the effectiveness of our framework on both familiar tasks involving new objects and novel but related tasks, highlighting the potential of LLMs in enhancing robotic system versatility and adaptability. Project website: https://object814.github.io/Task-Condition-With-LLM/