Beyond Model Adaptation at Test Time: A Survey

TL;DR

This survey comprehensively reviews test-time adaptation techniques in machine learning, analyzing over 400 papers, categorizing methods, and discussing their deployment in various modalities and future research directions.

Contribution

It provides the first systematic overview of test-time adaptation, categorizing methods and offering insights into their application across multiple data modalities.

Findings

Categorizes test-time adaptation methods into five groups.

Analyzes over 400 recent papers on the topic.

Discusses deployment in vision, video, 3D, and beyond.

Abstract

Machine learning algorithms have achieved remarkable success across various disciplines, use cases and applications, under the prevailing assumption that training and test samples are drawn from the same distribution. Consequently, these algorithms struggle and become brittle even when samples in the test distribution start to deviate from the ones observed during training. Domain adaptation and domain generalization have been studied extensively as approaches to address distribution shifts across test and train domains, but each has its limitations. Test-time adaptation, a recently emerging learning paradigm, combines the benefits of domain adaptation and domain generalization by training models only on source data and adapting them to target data during test-time inference. In this survey, we provide a comprehensive and systematic review on test-time adaptation, covering more than 400 recent papers. We structure our review by categorizing existing methods into five distinct categories based on what component of the method is adjusted for test-time adaptation: the model, the inference, the normalization, the sample, or the prompt, providing detailed analysis of each. We further discuss the various preparation and adaptation settings for methods within these categories, offering deeper insights into the effective deployment for the evaluation of distribution shifts and their real-world application in understanding images, video and 3D, as well as modalities beyond vision. We close the survey with an outlook on emerging research opportunities for test-time adaptation.