GuidedVLA: Specifying Task-Relevant Factors via Plug-and-Play Action Attention Specialization

TL;DR

GuidedVLA introduces a modular approach to vision-language-action models by supervising individual attention heads with auxiliary signals, enhancing focus on task-relevant factors and improving generalization in robot learning.

Contribution

The paper proposes a novel framework that explicitly guides action decoder components with auxiliary signals, enabling better focus on task-relevant features and improved robustness.

Findings

GuidedVLA improves success rates in simulation and real-robot experiments.

Specialized attention heads capture distinct task-relevant factors.

High-quality, decoupled features correlate with better task performance.

Abstract

Vision-Language-Action (VLA) models aim for general robot learning by aligning action as a modality within powerful Vision-Language Models (VLMs). Existing VLAs rely on end-to-end supervision to implicitly enable the action decoding process to learn task-relevant features. However, without explicit guidance, these models often overfit to spurious correlations, such as visual shortcuts or environmental noise, limiting their generalization. In this paper, we introduce GuidedVLA, a framework designed to manually guide the action generation to focus on task-relevant factors. Our core insight is to treat the action decoder not as a monolithic learner, but as an assembly of functional components. Individual attention heads are supervised by manually defined auxiliary signals to capture distinct factors. As an initial study, we instantiate this paradigm with three specialized heads: object grounding, spatial geometry, and temporal skill logic. Across simulation and real-robot experiments, GuidedVLA improves success rates in both in-domain and out-of-domain settings compared to strong VLA baselines. Finally, we show that the quality of these specialized factors correlates positively with task performance and that our mechanism yields decoupled, high-quality features. Our results suggest that explicitly guiding action-decoder learning is a promising direction for building more robust and general VLA models.