Rethinking Precision of Pseudo Label: Test-Time Adaptation via Complementary Learning

TL;DR

This paper introduces a complementary learning method for test-time adaptation that uses less probable categories as complementary labels to improve pseudo-label quality, leading to state-of-the-art results under distribution shifts.

Contribution

It proposes leveraging complementary labels to reduce pseudo-label noise in test-time adaptation, enhancing model robustness without source domain data.

Findings

Achieves state-of-the-art performance on multiple datasets

Effectively reduces pseudo-label noise

Improves test-time adaptation under distribution shifts

Abstract

In this work, we propose a novel complementary learning approach to enhance test-time adaptation (TTA), which has been proven to exhibit good performance on testing data with distribution shifts such as corruptions. In test-time adaptation tasks, information from the source domain is typically unavailable and the model has to be optimized without supervision for test-time samples. Hence, usual methods assign labels for unannotated data with the prediction by a well-trained source model in an unsupervised learning framework. Previous studies have employed unsupervised objectives, such as the entropy of model predictions, as optimization targets to effectively learn features for test-time samples. However, the performance of the model is easily compromised by the quality of pseudo-labels, since inaccuracies in pseudo-labels introduce noise to the model. Therefore, we propose to leverage the "less probable categories" to decrease the risk of incorrect pseudo-labeling. The complementary label is introduced to designate these categories. We highlight that the risk function of complementary labels agrees with their Vanilla loss formula under the conventional true label distribution. Experiments show that the proposed learning algorithm achieves state-of-the-art performance on different datasets and experiment settings.