REALM: Robust Entropy Adaptive Loss Minimization for Improved Single-Sample Test-Time Adaptation

TL;DR

This paper introduces REALM, a robust loss function framework for fully-test-time adaptation that enhances stability and accuracy in the presence of noisy samples during online adaptation, especially under distribution shifts.

Contribution

We propose a novel robust entropy adaptive loss framework, REALM, inspired by self-paced learning, to improve stability and accuracy in test-time adaptation with noisy samples.

Findings

REALM outperforms previous methods on CIFAR-10 and ImageNet-1K corruptions.

It achieves more stable and accurate online adaptation.

Demonstrates robustness to noisy test samples during adaptation.

Abstract

Fully-test-time adaptation (F-TTA) can mitigate performance loss due to distribution shifts between train and test data (1) without access to the training data, and (2) without knowledge of the model training procedure. In online F-TTA, a pre-trained model is adapted using a stream of test samples by minimizing a self-supervised objective, such as entropy minimization. However, models adapted with online using entropy minimization, are unstable especially in single sample settings, leading to degenerate solutions, and limiting the adoption of TTA inference strategies. Prior works identify noisy, or unreliable, samples as a cause of failure in online F-TTA. One solution is to ignore these samples, which can lead to bias in the update procedure, slow adaptation, and poor generalization. In this work, we present a general framework for improving robustness of F-TTA to these noisy samples, inspired by self-paced learning and robust loss functions. Our proposed approach, Robust Entropy Adaptive Loss Minimization (REALM), achieves better adaptation accuracy than previous approaches throughout the adaptation process on corruptions of CIFAR-10 and ImageNet-1K, demonstrating its effectiveness.