Global-Local Medical SAM Adaptor Based on Full Adaption

TL;DR

This paper introduces a novel global-local medical SAM adaptor that fully adapts the Segment Anything Model for improved medical image segmentation, demonstrating superior performance on melanoma datasets.

Contribution

The paper proposes a new global medical SAM adaptor with full adaptation and combines it with local adaptation to enhance segmentation performance.

Findings

GLMed-SA outperforms state-of-the-art methods on melanoma segmentation.

Full adaptation of SAM improves medical image segmentation.

Combining global and local adaptions yields superior results.

Abstract

Emerging of visual language models, such as the segment anything model (SAM), have made great breakthroughs in the field of universal semantic segmentation and significantly aid the improvements of medical image segmentation, in particular with the help of Medical SAM adaptor (Med-SA). However, Med-SA still can be improved, as it fine-tunes SAM in a partial adaption manner. To resolve this problem, we present a novel global medical SAM adaptor (GMed-SA) with full adaption, which can adapt SAM globally. We further combine GMed-SA and Med-SA to propose a global-local medical SAM adaptor (GLMed-SA) to adapt SAM both globally and locally. Extensive experiments have been performed on the challenging public 2D melanoma segmentation dataset. The results show that GLMed-SA outperforms several state-of-the-art semantic segmentation methods on various evaluation metrics, demonstrating the superiority of our methods.