COME: Test-time adaption by Conservatively Minimizing Entropy

TL;DR

COME introduces a conservative entropy minimization approach for test-time adaptation, modeling uncertainty with a Dirichlet prior to improve model stability and accuracy on open-world data.

Contribution

It proposes a novel method that replaces traditional entropy minimization with a Dirichlet-based regularization to address overconfidence in test-time adaptation.

Findings

Achieves state-of-the-art accuracy improvements up to 34.5%.

Reduces false positive rate by up to 15.1%.

Enhances stability and uncertainty estimation in various TTA settings.

Abstract

Machine learning models must continuously self-adjust themselves for novel data distribution in the open world. As the predominant principle, entropy minimization (EM) has been proven to be a simple yet effective cornerstone in existing test-time adaption (TTA) methods. While unfortunately its fatal limitation (i.e., overconfidence) tends to result in model collapse. For this issue, we propose to Conservatively Minimize the Entropy (COME), which is a simple drop-in replacement of traditional EM to elegantly address the limitation. In essence, COME explicitly models the uncertainty by characterizing a Dirichlet prior distribution over model predictions during TTA. By doing so, COME naturally regularizes the model to favor conservative confidence on unreliable samples. Theoretically, we provide a preliminary analysis to reveal the ability of COME in enhancing the optimization stability by introducing a data-adaptive lower bound on the entropy. Empirically, our method achieves state-of-the-art performance on commonly used benchmarks, showing significant improvements in terms of classification accuracy and uncertainty estimation under various settings including standard, life-long and open-world TTA, i.e., up to $34.5\%$ improvement on accuracy and $15.1\%$ on false positive rate.