Exploiting DINOv3-Based Self-Supervised Features for Robust Few-Shot Medical Image Segmentation

TL;DR

This paper introduces DINO-AugSeg, a framework that enhances self-supervised features from DINOv3 for robust few-shot medical image segmentation across multiple modalities, using wavelet augmentation and contextual fusion.

Contribution

It presents a novel combination of wavelet-based augmentation and contextual feature fusion to improve DINOv3's applicability to medical images in few-shot scenarios.

Findings

Outperforms existing methods on six benchmarks

Effective across five different imaging modalities

Demonstrates robustness with limited training samples

Abstract

Deep learning-based automatic medical image segmentation plays a critical role in clinical diagnosis and treatment planning but remains challenging in few-shot scenarios due to the scarcity of annotated training data. Recently, self-supervised foundation models such as DINOv3, which were trained on large natural image datasets, have shown strong potential for dense feature extraction that can help with the few-shot learning challenge. Yet, their direct application to medical images is hindered by domain differences. In this work, we propose DINO-AugSeg, a novel framework that leverages DINOv3 features to address the few-shot medical image segmentation challenge. Specifically, we introduce WT-Aug, a wavelet-based feature-level augmentation module that enriches the diversity of DINOv3-extracted features by perturbing frequency components, and CG-Fuse, a contextual information-guided fusion module that exploits cross-attention to integrate semantic-rich low-resolution features with spatially detailed high-resolution features. Extensive experiments on six public benchmarks spanning five imaging modalities, including MRI, CT, ultrasound, endoscopy, and dermoscopy, demonstrate that DINO-AugSeg consistently outperforms existing methods under limited-sample conditions. The results highlight the effectiveness of incorporating wavelet-domain augmentation and contextual fusion for robust feature representation, suggesting DINO-AugSeg as a promising direction for advancing few-shot medical image segmentation. Code and data will be made available on https://github.com/apple1986/DINO-AugSeg.