A3-TTA: Adaptive Anchor Alignment Test-Time Adaptation for Image Segmentation

TL;DR

A3-TTA introduces an anchor-guided pseudo-labeling framework for test-time adaptation in image segmentation, significantly improving performance under domain shifts without source data access.

Contribution

It proposes a novel anchor-guided supervision method with a self-adaptive strategy to enhance stability and accuracy in test-time domain adaptation for segmentation.

Findings

Improves Dice scores by 10.40 to 17.68 percentage points.

Outperforms state-of-the-art TTA methods across multiple datasets.

Maintains high performance in continual TTA with anti-forgetting capabilities.

Abstract

Test-Time Adaptation (TTA) offers a practical solution for deploying image segmentation models under domain shift without accessing source data or retraining. Among existing TTA strategies, pseudo-label-based methods have shown promising performance. However, they often rely on perturbation-ensemble heuristics (e.g., dropout sampling, test-time augmentation, Gaussian noise), which lack distributional grounding and yield unstable training signals. This can trigger error accumulation and catastrophic forgetting during adaptation. To address this, we propose \textbf{A3-TTA}, a TTA framework that constructs reliable pseudo-labels through anchor-guided supervision. Specifically, we identify well-predicted target domain images using a class compact density metric, under the assumption that confident predictions imply distributional proximity to the source domain. These anchors serve as stable references to guide pseudo-label generation, which is further regularized via semantic consistency and boundary-aware entropy minimization. Additionally, we introduce a self-adaptive exponential moving average strategy to mitigate label noise and stabilize model update during adaptation. Evaluated on both multi-domain medical images (heart structure and prostate segmentation) and natural images, A3-TTA significantly improves average Dice scores by 10.40 to 17.68 percentage points compared to the source model, outperforming several state-of-the-art TTA methods under different segmentation model architectures. A3-TTA also excels in continual TTA, maintaining high performance across sequential target domains with strong anti-forgetting ability. The code will be made publicly available at https://github.com/HiLab-git/A3-TTA.