AugVLA-3D: Depth-Driven Feature Augmentation for Vision-Language-Action Models

TL;DR

This paper introduces AugVLA-3D, a depth-driven feature augmentation framework for vision-language-action models that enhances 3D understanding and control in robotic perception by integrating depth estimation and action priors.

Contribution

It proposes a novel method combining depth estimation and an action assistant module to improve 3D feature representation and robustness in VLA models.

Findings

Improved action prediction accuracy in complex 3D environments.

Enhanced perception robustness in geometrically ambiguous scenarios.

Better generalization of VLA models through depth-driven data augmentation.

Abstract

Vision-Language-Action (VLA) models have recently achieved remarkable progress in robotic perception and control, yet most existing approaches primarily rely on VLM trained using 2D images, which limits their spatial understanding and action grounding in complex 3D environments. To address this limitation, we propose a novel framework that integrates depth estimation into VLA models to enrich 3D feature representations. Specifically, we employ a depth estimation baseline called VGGT to extract geometry-aware 3D cues from standard RGB inputs, enabling efficient utilization of existing large-scale 2D datasets while implicitly recovering 3D structural information. To further enhance the reliability of these depth-derived features, we introduce a new module called action assistant, which constrains the learned 3D representations with action priors and ensures their consistency with downstream control tasks. By fusing the enhanced 3D features with conventional 2D visual tokens, our approach significantly improves the generalization ability and robustness of VLA models. Experimental results demonstrate that the proposed method not only strengthens perception in geometrically ambiguous scenarios but also leads to superior action prediction accuracy. This work highlights the potential of depth-driven data augmentation and auxiliary expert supervision for bridging the gap between 2D observations and 3D-aware decision-making in robotic systems.