IT$^3$: Idempotent Test-Time Training

TL;DR

IT$^3$ introduces a simple, universal test-time training method that enforces idempotence to adapt models to distribution shifts without auxiliary tasks, improving out-of-distribution performance across diverse domains.

Contribution

The paper proposes a novel idempotence-based test-time training approach that eliminates the need for auxiliary tasks, enabling effective adaptation to distribution shifts in a wide range of models and domains.

Findings

IT$^3$ outperforms existing TTT methods across multiple tasks.

Enforcing idempotence improves model robustness to distribution shifts.

The approach is simple, model-agnostic, and widely applicable.

Abstract

Deep learning models often struggle when deployed in real-world settings due to distribution shifts between training and test data. While existing approaches like domain adaptation and test-time training (TTT) offer partial solutions, they typically require additional data or domain-specific auxiliary tasks. We present Idempotent Test-Time Training (IT$^3$), a novel approach that enables on-the-fly adaptation to distribution shifts using only the current test instance, without any auxiliary task design. Our key insight is that enforcing idempotence -- where repeated applications of a function yield the same result -- can effectively replace domain-specific auxiliary tasks used in previous TTT methods. We theoretically connect idempotence to prediction confidence and demonstrate that minimizing the distance between successive applications of our model during inference leads to improved out-of-distribution performance. Extensive experiments across diverse domains (including image classification, aerodynamics prediction, and aerial segmentation) and architectures (MLPs, CNNs, GNNs) show that IT$^3$ consistently outperforms existing approaches while being simpler and more widely applicable. Our results suggest that idempotence provides a universal principle for test-time adaptation that generalizes across domains and architectures.