Adaptive Cascading Network for Continual Test-Time Adaptation

TL;DR

This paper introduces an adaptive cascading network that enhances continual test-time adaptation by jointly updating feature extractors and classifiers, employing meta-learning for rapid adaptation across various tasks.

Contribution

It proposes a novel cascading paradigm with meta-learning pre-training to improve long-term adaptation and reduce task interference during test-time adaptation.

Findings

Outperforms existing methods in image, text, and speech tasks.

Achieves faster adaptation with limited unlabelled data.

Introduces new metrics for measuring adaptation effectiveness.

Abstract

We study the problem of continual test-time adaption where the goal is to adapt a source pre-trained model to a sequence of unlabelled target domains at test time. Existing methods on test-time training suffer from several limitations: (1) Mismatch between the feature extractor and classifier; (2) Interference between the main and self-supervised tasks; (3) Lack of the ability to quickly adapt to the current distribution. In light of these challenges, we propose a cascading paradigm that simultaneously updates the feature extractor and classifier at test time, mitigating the mismatch between them and enabling long-term model adaptation. The pre-training of our model is structured within a meta-learning framework, thereby minimizing the interference between the main and self-supervised tasks and encouraging fast adaptation in the presence of limited unlabelled data. Additionally, we introduce innovative evaluation metrics, average accuracy and forward transfer, to effectively measure the model's adaptation capabilities in dynamic, real-world scenarios. Extensive experiments and ablation studies demonstrate the superiority of our approach in a range of tasks including image classification, text classification, and speech recognition.