Beyond Entropy: Region Confidence Proxy for Wild Test-Time Adaptation

TL;DR

This paper introduces ReCAP, a novel region confidence proxy for wild test-time adaptation, addressing limitations of entropy minimization and enabling efficient, robust adaptation in unseen domains with data scarcity.

Contribution

ReCAP provides a tractable, upper-bounded proxy for region confidence, improving adaptation efficiency and robustness in wild test-time scenarios.

Findings

ReCAP outperforms existing WTTA methods across multiple datasets.

Region confidence is a more effective measure than entropy for adaptation.

The proposed probabilistic region modeling captures semantic changes effectively.

Abstract

Wild Test-Time Adaptation (WTTA) is proposed to adapt a source model to unseen domains under extreme data scarcity and multiple shifts. Previous approaches mainly focused on sample selection strategies, while overlooking the fundamental problem on underlying optimization. Initially, we critically analyze the widely-adopted entropy minimization framework in WTTA and uncover its significant limitations in noisy optimization dynamics that substantially hinder adaptation efficiency. Through our analysis, we identify region confidence as a superior alternative to traditional entropy, however, its direct optimization remains computationally prohibitive for real-time applications. In this paper, we introduce a novel region-integrated method ReCAP that bypasses the lengthy process. Specifically, we propose a probabilistic region modeling scheme that flexibly captures semantic changes in embedding space. Subsequently, we develop a finite-to-infinite asymptotic approximation that transforms the intractable region confidence into a tractable and upper-bounded proxy. These innovations significantly unlock the overlooked potential dynamics in local region in a concise solution. Our extensive experiments demonstrate the consistent superiority of ReCAP over existing methods across various datasets and wild scenarios.