Test-Time Training for Visual Foresight Vision-Language-Action Models

TL;DR

This paper introduces T3VF, a test-time training method for visual foresight vision-language-action models that enhances robustness to out-of-distribution shifts without altering the model architecture.

Contribution

The paper proposes a novel test-time training approach with adaptive update filtering to improve OOD robustness in VF-VLA models, avoiding architectural changes.

Findings

T3VF reduces OOD vulnerability in VF-VLA models.

It achieves this with minimal additional inference cost.

No extra modules or architecture modifications are needed.

Abstract

Visual Foresight VLA (VF-VLA) has become a prominent architectural choice in the recent VLA due to its impressive performance. Nevertheless, the inherent design of VF-VLA makes it particularly vulnerable to out-of-distribution (OOD) shifts. Because the quality of action directly depends on the accuracy of the predicted future visual information, OOD conditions affect both stages at once. To address this vulnerability, we propose Test-Time Training Visual Foresight VLA ($T^3$VF), a test-time training approach motivated by the observation that the predicted future image and its subsequent observation form a natural supervision pair. To further address the practical challenges that arise from indiscriminate test-time updates, we introduce an adaptive update filtering mechanism. Empirically, $T^3$VF mitigates the OOD vulnerability of VF-VLA at a modest additional inference cost, without requiring any architectural modification or auxiliary modules.