Feature-Based Instance Neighbor Discovery: Advanced Stable Test-Time Adaptation in Dynamic World

TL;DR

This paper introduces FIND, a novel test-time adaptation method that leverages feature clustering and selective normalization to improve neural network robustness under dynamic distribution shifts, outperforming existing approaches.

Contribution

FIND combines layer-wise feature disentanglement with adaptive batch normalization and layer selection, addressing distribution shift challenges more effectively than prior global normalization methods.

Findings

FIND achieves 30% accuracy improvement in dynamic test scenarios.

The method maintains high computational efficiency.

It effectively handles multiple distribution shifts within test batches.

Abstract

Despite progress, deep neural networks still suffer performance declines under distribution shifts between training and test domains, leading to a substantial decrease in Quality of Experience (QoE) for applications. Existing test-time adaptation (TTA) methods are challenged by dynamic, multiple test distributions within batches. We observe that feature distributions across different domains inherently cluster into distinct groups with varying means and variances. This divergence reveals a critical limitation of previous global normalization strategies in TTA, which inevitably distort the original data characteristics. Based on this insight, we propose Feature-based Instance Neighbor Discovery (FIND), which comprises three key components: Layer-wise Feature Disentanglement (LFD), Feature Aware Batch Normalization (FABN) and Selective FABN (S-FABN). LFD stably captures features with similar distributions at each layer by constructing graph structures. While FABN optimally combines source statistics with test-time distribution specific statistics for robust feature representation. Finally, S-FABN determines which layers require feature partitioning and which can remain unified, thereby enhancing inference efficiency. Extensive experiments demonstrate that FIND significantly outperforms existing methods, achieving a 30\% accuracy improvement in dynamic scenarios while maintaining computational efficiency.