Autoregressive Action Sequence Learning for Robotic Manipulation

TL;DR

This paper introduces a novel autoregressive sequence modeling approach for robotic manipulation, enabling a universal, efficient, and flexible policy architecture that outperforms existing methods across diverse tasks.

Contribution

We propose Chunking Causal Transformer (CCT) and Autoregressive Policy (ARP), enabling hybrid action sequences and improved performance across various robotic manipulation environments.

Findings

ARP matches or outperforms environment-specific state-of-the-art methods

Our approach is more computationally efficient and has fewer parameters

Effective across diverse tasks including Push-T, ALOHA, and RLBench

Abstract

Designing a universal policy architecture that performs well across diverse robots and task configurations remains a key challenge. In this work, we address this by representing robot actions as sequential data and generating actions through autoregressive sequence modeling. Existing autoregressive architectures generate end-effector waypoints sequentially as word tokens in language modeling, which are limited to low-frequency control tasks. Unlike language, robot actions are heterogeneous and often include continuous values -- such as joint positions, 2D pixel coordinates, and end-effector poses -- which are not easily suited for language-based modeling. Based on this insight, we introduce a straightforward enhancement: we extend causal transformers' single-token prediction to support predicting a variable number of tokens in a single step through our Chunking Causal Transformer (CCT). This enhancement enables robust performance across diverse tasks of various control frequencies, greater efficiency by having fewer autoregression steps, and lead to a hybrid action sequence design by mixing different types of actions and using a different chunk size for each action type. Based on CCT, we propose the Autoregressive Policy (ARP) architecture, which solves manipulation tasks by generating hybrid action sequences. We evaluate ARP across diverse robotic manipulation environments, including Push-T, ALOHA, and RLBench, and show that ARP, as a universal architecture, matches or outperforms the environment-specific state-of-the-art in all tested benchmarks, while being more efficient in computation and parameter sizes. Videos of our real robot demonstrations, all source code and the pretrained models of ARP can be found at http://github.com/mlzxy/arp.