Order-Aware Test-Time Adaptation: Leveraging Temporal Dynamics for Robust Streaming Inference

TL;DR

OATTA introduces a novel order-aware test-time adaptation method that leverages temporal dynamics through Bayesian estimation and a likelihood-ratio gate, significantly improving model robustness across diverse streaming tasks.

Contribution

This work presents a lightweight, model-agnostic framework that incorporates temporal information into test-time adaptation, addressing the limitations of order-agnostic methods.

Findings

Consistently improves accuracy by up to 6.35% across tasks.

Effectively models temporal dynamics for robust streaming inference.

Applicable to diverse domains like image, signal, and language analysis.

Abstract

Test-Time Adaptation (TTA) enables pre-trained models to adjust to distribution shift by learning from unlabeled test-time streams. However, existing methods typically treat these streams as independent samples, overlooking the supervisory signal inherent in temporal dynamics. To address this, we introduce Order-Aware Test-Time Adaptation (OATTA). We formulate test-time adaptation as a gradient-free recursive Bayesian estimation task, using a learned dynamic transition matrix as a temporal prior to refine the base model's predictions. To ensure safety in weakly structured streams, we introduce a likelihood-ratio gate (LLR) that reverts to the base predictor when temporal evidence is absent. OATTA is a lightweight, model-agnostic module that incurs negligible computational overhead. Extensive experiments across image classification, wearable and physiological signal analysis, and language sentiment analysis demonstrate its universality; OATTA consistently boosts established baselines, improving accuracy by up to 6.35%. Our findings establish that modeling temporal dynamics provides a critical, orthogonal signal beyond standard order-agnostic TTA approaches.