Boosting Medical Image Classification with Segmentation Foundation Model

TL;DR

This paper introduces SAMAug-C, a novel augmentation method using the Segment Anything Model to enhance medical image classification by generating augmented datasets and a framework that combines raw and augmented images, improving classification accuracy.

Contribution

The study presents a new SAM-based augmentation technique and a dual-input framework to improve medical image classification performance.

Findings

Enhanced classification accuracy on three public datasets.

Effective augmentation of datasets using SAMAug-C.

Complementary processing of raw and augmented images improves results.

Abstract

The Segment Anything Model (SAM) exhibits impressive capabilities in zero-shot segmentation for natural images. Recently, SAM has gained a great deal of attention for its applications in medical image segmentation. However, to our best knowledge, no studies have shown how to harness the power of SAM for medical image classification. To fill this gap and make SAM a true ``foundation model'' for medical image analysis, it is highly desirable to customize SAM specifically for medical image classification. In this paper, we introduce SAMAug-C, an innovative augmentation method based on SAM for augmenting classification datasets by generating variants of the original images. The augmented datasets can be used to train a deep learning classification model, thereby boosting the classification performance. Furthermore, we propose a novel framework that simultaneously processes raw and SAMAug-C augmented image input, capitalizing on the complementary information that is offered by both. Experiments on three public datasets validate the effectiveness of our new approach.