SegTTA: Training-Free Test-Time Augmentation for Zero-Shot Medical Imaging Segmentation

TL;DR

SegTTA is a training-free test-time augmentation framework that enhances zero-shot medical image segmentation by combining specific augmentations and voting, improving performance across various datasets.

Contribution

It introduces a novel, training-free test-time augmentation method that improves segmentation accuracy without retraining models, using multiple augmentations and weighted voting.

Findings

Consistent performance improvements across three diverse datasets.

Large organs benefit from intensity augmentations, small lesions from noise.

Achieves higher mIoU and aIoU, and reduces HD95 on hepatic vessel dataset.

Abstract

Increasingly advanced data augmentation techniques have greatly aided clinical medical research, increasing data diversity and improving model generalization capabilities. Although most current basic models exhibit strong generalization abilities, image quality varies due to differences in equipment and operators. To address these challenges, we present SegTTA, a framework that improves medical image segmentation without model retraining by combining four augmentations (Gamma correction, Contrast enhancement, Gaussian blur, Gaussian noise) with weighted voting across multiple MedSAM2 checkpoints. Experiments demonstrate consistent improvements across three diverse datasets: healthy uterus segmentation, uterine myoma detection, and multi class hepatic structure segmentation. Ablation studies reveal that large organs benefit from intensity augmentations while small lesions require noise augmentations. The voting threshold controls the coverage precision trade off, enabling task specific optimization for different clinical requirements. Ultimately, on a multiclass hepatic vessel dataset, compared to MedSAM2 baselines, our method achieves an increase of 1.6 in mIoU and 1.9 in aIoU, along with a reduction of approximately 2.0 in HD95. Code will be available at https://github.com/AIGeeksGroup/SegTTA.