Selective Perception for Robot: Task-Aware Attention in Multimodal VLA

TL;DR

This paper introduces a dynamic, task-aware attention framework for robot vision-language-action models that improves efficiency and robustness by selectively processing relevant visual inputs in real-time, inspired by human perception.

Contribution

It proposes a lightweight adaptive routing architecture for real-time, task-dependent multimodal data fusion in robotic perception, reducing unnecessary computation and noise.

Findings

Significant improvements in inference efficiency.

Enhanced control performance in robotic manipulation.

Validated effectiveness in real-world scenarios.

Abstract

In robotics, Vision-Language-Action (VLA) models that integrate diverse multimodal signals from multi-view inputs have emerged as an effective approach. However, most prior work adopts static fusion that processes all visual inputs uniformly, which incurs unnecessary computational overhead and allows task-irrelevant background information to act as noise. Inspired by the principles of human active perception, we propose a dynamic information fusion framework designed to maximize the efficiency and robustness of VLA models. Our approach introduces a lightweight adaptive routing architecture that analyzes the current text prompt and observations from a wrist-mounted camera in real-time to predict the task-relevance of multiple camera views. By conditionally attenuating computations for views with low informational utility and selectively providing only essential visual features to the policy network, Our framework achieves computation efficiency proportional to task relevance. Furthermore, to efficiently secure large-scale annotation data for router training, we established an automated labeling pipeline utilizing Vision-Language Models (VLMs) to minimize data collection and annotation costs. Experimental results in real-world robotic manipulation scenarios demonstrate that the proposed approach achieves significant improvements in both inference efficiency and control performance compared to existing VLA models, validating the effectiveness and practicality of dynamic information fusion in resource-constrained, real-time robot control environments.