When and Where to Reset Matters for Long-Term Test-Time Adaptation

TL;DR

This paper introduces an adaptive reset strategy for long-term test-time adaptation that selectively resets parts of the model, uses importance-aware regularization to retain crucial knowledge, and dynamically adjusts adaptation to improve robustness under domain shifts.

Contribution

It proposes a novel adaptive and selective reset method with importance-aware regularization and on-the-fly adjustment, addressing limitations of previous reset strategies in long-term TTA.

Findings

Significantly reduces model collapse in long-term TTA

Improves adaptation performance under challenging domain shifts

Outperforms existing reset strategies on benchmark datasets

Abstract

When continual test-time adaptation (TTA) persists over the long term, errors accumulate in the model and further cause it to predict only a few classes for all inputs, a phenomenon known as model collapse. Recent studies have explored reset strategies that completely erase these accumulated errors. However, their periodic resets lead to suboptimal adaptation, as they occur independently of the actual risk of collapse. Moreover, their full resets cause catastrophic loss of knowledge acquired over time, even though such knowledge could be beneficial in the future. To this end, we propose (1) an Adaptive and Selective Reset (ASR) scheme that dynamically determines when and where to reset, (2) an importance-aware regularizer to recover essential knowledge lost due to reset, and (3) an on-the-fly adaptation adjustment scheme to enhance adaptability under challenging domain shifts. Extensive experiments across long-term TTA benchmarks demonstrate the effectiveness of our approach, particularly under challenging conditions. Our code is available at https://github.com/YonseiML/asr.