AnoleVLA: Lightweight Vision-Language-Action Model with Deep State Space Models for Mobile Manipulation

TL;DR

AnoleVLA is a lightweight vision-language-action model that employs deep state space models to enable efficient, real-time robotic manipulation in resource-constrained environments, outperforming larger models in success rate and speed.

Contribution

The paper introduces AnoleVLA, a novel lightweight VLA using deep state space models for efficient multimodal processing in robotic manipulation tasks.

Findings

AnoleVLA outperforms large-scale VLA by 21 points in success rate.

AnoleVLA achieves three times faster inference speed.

The model is effective in both simulation and real-world environments.

Abstract

In this study, we address the problem of language-guided robotic manipulation, where a robot is required to manipulate a wide range of objects based on visual observations and natural language instructions. This task is essential for service robots that operate in human environments, and requires safety, efficiency, and task-level generality. Although Vision-Language-Action models (VLAs) have demonstrated strong performance for this task, their deployment in resource-constrained environments remains challenging because of the computational cost of standard transformer backbones. To overcome this limitation, we propose AnoleVLA, a lightweight VLA that uses a deep state space model to process multimodal sequences efficiently. The model leverages its lightweight and fast sequential state modeling to process visual and textual inputs, which allows the robot to generate trajectories efficiently. We evaluated the proposed method in both simulation and physical experiments. Notably, in real-world evaluations, AnoleVLA outperformed a representative large-scale VLA by 21 points for the task success rate while achieving an inference speed approximately three times faster.