TTF-VLA: Temporal Token Fusion via Pixel-Attention Integration for Vision-Language-Action Models

TL;DR

The paper introduces TTF-VLA, a novel temporal token fusion method that enhances vision-language-action models by integrating historical visual data, leading to improved performance and robustness in robotic manipulation tasks.

Contribution

It presents a training-free, selective temporal token fusion approach that leverages pixel difference and attention relevance, improving VLA model accuracy across various environments.

Findings

Achieved 4.0% average improvement on LIBERO dataset

Demonstrated 4.8% relative improvement on SimplerEnv

Real robot tasks saw an 8.7% relative performance boost

Abstract

Vision-Language-Action (VLA) models process visual inputs independently at each timestep, discarding valuable temporal information inherent in robotic manipulation tasks. This frame-by-frame processing makes models vulnerable to visual noise while ignoring the substantial coherence between consecutive frames in manipulation sequences. We propose Temporal Token Fusion (TTF), a training-free approach that intelligently integrates historical and current visual representations to enhance VLA inference quality. Our method employs dual-dimension detection combining efficient grayscale pixel difference analysis with attention-based semantic relevance assessment, enabling selective temporal token fusion through hard fusion strategies and keyframe anchoring to prevent error accumulation. Comprehensive experiments across LIBERO, SimplerEnv, and real robot tasks demonstrate consistent improvements: 4.0 percentage points average on LIBERO (72.4\% vs 68.4\% baseline), cross-environment validation on SimplerEnv (4.8\% relative improvement), and 8.7\% relative improvement on real robot tasks. Our approach proves model-agnostic, working across OpenVLA and VLA-Cache architectures. Notably, TTF reveals that selective Query matrix reuse in attention mechanisms enhances rather than compromises performance, suggesting promising directions for direct KQV matrix reuse strategies that achieve computational acceleration while improving task success rates.