TTN: A Domain-Shift Aware Batch Normalization in Test-Time Adaptation

TL;DR

This paper introduces TTN, a new batch normalization method for test-time adaptation that interpolates between conventional and transductive batch normalization, improving robustness across domain shifts and batch sizes.

Contribution

The paper presents a novel test-time normalization (TTN) method that balances CBN and TBN, addressing their trade-off and enhancing domain-shift robustness in various scenarios.

Findings

TTN outperforms existing methods across multiple benchmarks.

TTN maintains performance with small batch sizes.

Integrating TTN with other adaptation methods yields state-of-the-art results.

Abstract

This paper proposes a novel batch normalization strategy for test-time adaptation. Recent test-time adaptation methods heavily rely on the modified batch normalization, i.e., transductive batch normalization (TBN), which calculates the mean and the variance from the current test batch rather than using the running mean and variance obtained from the source data, i.e., conventional batch normalization (CBN). Adopting TBN that employs test batch statistics mitigates the performance degradation caused by the domain shift. However, re-estimating normalization statistics using test data depends on impractical assumptions that a test batch should be large enough and be drawn from i.i.d. stream, and we observed that the previous methods with TBN show critical performance drop without the assumptions. In this paper, we identify that CBN and TBN are in a trade-off relationship and present a new test-time normalization (TTN) method that interpolates the statistics by adjusting the importance between CBN and TBN according to the domain-shift sensitivity of each BN layer. Our proposed TTN improves model robustness to shifted domains across a wide range of batch sizes and in various realistic evaluation scenarios. TTN is widely applicable to other test-time adaptation methods that rely on updating model parameters via backpropagation. We demonstrate that adopting TTN further improves their performance and achieves state-of-the-art performance in various standard benchmarks.