DexVLA: Vision-Language Model with Plug-In Diffusion Expert for General Robot Control

TL;DR

DexVLA introduces a scalable diffusion-based action expert and an embodiment curriculum to enhance vision-language models for versatile, long-horizon robot tasks, enabling rapid adaptation and superior performance across diverse robot embodiments.

Contribution

The paper presents DexVLA, a novel framework with a diffusion-based action expert and curriculum learning for improved generalization and efficiency in vision-language robot control.

Findings

DexVLA outperforms state-of-the-art models in diverse robot tasks.

It enables dexterous skill learning with limited data.

The model adapts to new embodiments without task-specific training.

Abstract

Enabling robots to perform diverse tasks across varied environments is a central challenge in robot learning. While vision-language-action (VLA) models have shown promise for generalizable robot skills, realizing their full potential requires addressing limitations in action representation and efficient training. Current VLA models often focus on scaling the vision-language model (VLM) component, while the action space representation remains a critical bottleneck. This paper introduces DexVLA, a novel framework designed to enhance the efficiency and generalization capabilities of VLAs for complex, long-horizon tasks across diverse robot embodiments. DexVLA features a novel diffusion-based action expert, scaled to one billion parameters, designed for cross-embodiment learning. A novel embodiment curriculum learning strategy facilitates efficient training: (1) pre-training the diffusion expert that is separable from the VLA on cross-embodiment data, (2) aligning the VLA model to specific embodiments, and (3) post-training for rapid adaptation to new tasks. We conduct comprehensive experiments across multiple embodiments, including single-arm, bimanual, and dexterous hand, demonstrating DexVLA's adaptability to challenging tasks without task-specific adaptation, its ability to learn dexterous skills on novel embodiments with limited data, and its capacity to complete complex, long-horizon tasks using only direct language prompting, such as laundry folding. In all settings, our method demonstrates superior performance compared to state-of-the-art models like Octo, OpenVLA, and Diffusion Policy.