ReservoirTTA: Prolonged Test-time Adaptation for Evolving and Recurring Domains

TL;DR

ReservoirTTA is a framework for prolonged test-time adaptation that maintains a reservoir of specialized models to handle evolving and recurring domain shifts, improving robustness and accuracy.

Contribution

It introduces a multi-model ensemble approach with online domain detection and routing, addressing limitations of single-model TTA like catastrophic forgetting.

Findings

Outperforms state-of-the-art methods on multiple benchmarks.

Effectively handles recurring and evolving domain shifts.

Maintains stable performance over prolonged adaptation periods.

Abstract

This paper introduces ReservoirTTA, a novel plug-in framework designed for prolonged test-time adaptation (TTA) in scenarios where the test domain continuously shifts over time, including cases where domains recur or evolve gradually. At its core, ReservoirTTA maintains a reservoir of domain-specialized models -- an adaptive test-time model ensemble -- that both detects new domains via online clustering over style features of incoming samples and routes each sample to the appropriate specialized model, and thereby enables domain-specific adaptation. This multi-model strategy overcomes key limitations of single model adaptation, such as catastrophic forgetting, inter-domain interference, and error accumulation, ensuring robust and stable performance on sustained non-stationary test distributions. Our theoretical analysis reveals key components that bound parameter variance and prevent model collapse, while our plug-in TTA module mitigates catastrophic forgetting of previously encountered domains. Extensive experiments on scene-level corruption benchmarks (ImageNet-C, CIFAR-10/100-C), object-level style shifts (DomainNet-126, PACS), and semantic segmentation (Cityscapes->ACDC) covering recurring and continuously evolving domain shifts -- show that ReservoirTTA substantially improves adaptation accuracy and maintains stable performance across prolonged, recurring shifts, outperforming state-of-the-art methods. Our code is publicly available at https://github.com/LTS5/ReservoirTTA.