Fine-Tuned Segment Anything Model with Low-Rank Adaptation for Chest X-Ray Images

TL;DR

This paper shows that fine-tuning the Segment Anything Model with low-rank adaptation improves chest X-ray segmentation accuracy and efficiency.

Contribution

The novel approach combines low-rank adaptation with SAM for efficient and effective medical image segmentation.

Findings

Fine-tuned SAM with LoRA outperforms zero-shot SAM methods in segmentation accuracy.

The proposed method also surpasses CNN baselines like U-Net and DeepLabv3+.

LoRA reduces memory and computational needs while preserving pre-trained SAM knowledge.

Abstract

Background: This paper investigates the use of the Segment Anything Model (SAM) for chest X-ray (CXR) image segmentation, with a focus on improving its performance using low-rank adaptation (LoRA). Methods: We evaluate three versions of SAM: two zero-shot methods (using coordinate and bounding box prompts) and a fine-tuned SAM using LoRA. To support these approaches, we also trained two standard convolutional neural networks (CNNs), U-Net and DeepLabv3+, to generate draft lung segmentations that serve as input prompts for the SAM methods. Our fine-tuning approach uses LoRA to add lightweight trainable adapters within the Transformer blocks of the SAM, allowing only a small subset of parameters to be updated. The rest of the SAM remains frozen, helping preserve its pre-trained knowledge while reducing memory and computational needs. We tested all models on a dataset of CXR images labeled for COVID-19, viral pneumonia, and normal cases. Results: Results show that fine-tuned SAM with LoRA outperforms zero-shot SAM methods and CNN baselines in terms of segmentation accuracy and efficiency. Conclusions: This demonstrates the potential of combining LoRA with SAM for practical and effective medical image segmentation.

Figures