TTA-Vid: Generalized Test-Time Adaptation for Video Reasoning

TL;DR

TTA-Vid introduces a test-time reinforcement learning approach for video reasoning that adapts a pretrained model on incoming video data without labels, improving performance across tasks.

Contribution

It proposes a novel test-time adaptation method combining reasoning and reward-based updates, enabling models to generalize across datasets without additional training.

Findings

Outperforms state-of-the-art methods on multiple video reasoning tasks.

Requires no ground-truth annotations during adaptation.

Generalizes effectively across different datasets.

Abstract

Recent video reasoning models have shown strong results on temporal and multimodal understanding, yet they depend on large-scale supervised data and multi-stage training pipelines, making them costly to train and difficult to adapt to new domains. In this work, we leverage the paradigm of Test-Time Reinforcement Learning on video-language data to allow for adapting a pretrained model to incoming video samples at test-time without explicit labels. The proposed test-time adaptation for video approach (TTA-Vid) combines two components that work simultaneously: (1) a test-time adaptation that performs step-by-step reasoning at inference time on multiple frame subsets. We then use a batch-aware frequency-based reward computed across different frame subsets as pseudo ground truth to update the model. It shows that the resulting model trained on a single batch or even a single sample from a dataset, is able to generalize at test-time to the whole dataset and even across datasets. Because the adaptation occurs entirely at test time, our method requires no ground-truth annotations or dedicated training splits. Additionally, we propose a multi-armed bandit strategy for adaptive frame selection that learns to prioritize informative frames, guided by the same reward formulation. Our evaluation shows that TTA-Vid yields consistent improvements across various video reasoning tasks and is able to outperform current state-of-the-art methods trained on large-scale data. This highlights the potential of test-time reinforcement learning for temporal multimodal understanding.