Assessing the generalization performance of SAM for ureteroscopy scene understanding

TL;DR

This paper evaluates the Segment Anything Model (SAM) for kidney stone segmentation in ureteroscopy images, demonstrating its superior generalization ability over traditional models like U-Net, especially on unseen data.

Contribution

It introduces SAM as a robust alternative for ureteroscopy scene understanding, highlighting its improved generalization compared to existing segmentation models.

Findings

SAM outperforms U-Net variants on out-of-distribution data

SAM achieves comparable accuracy on in-distribution data

SAM shows significantly better adaptability and efficiency

Abstract

The segmentation of kidney stones is regarded as a critical preliminary step to enable the identification of urinary stone types through machine- or deep-learning-based approaches. In urology, manual segmentation is considered tedious and impractical due to the typically large scale of image databases and the continuous generation of new data. In this study, the potential of the Segment Anything Model (SAM) -- a state-of-the-art deep learning framework -- is investigated for the automation of kidney stone segmentation. The performance of SAM is evaluated in comparison to traditional models, including U-Net, Residual U-Net, and Attention U-Net, which, despite their efficiency, frequently exhibit limitations in generalizing to unseen datasets. The findings highlight SAM's superior adaptability and efficiency. While SAM achieves comparable performance to U-Net on in-distribution data (Accuracy: 97.68 + 3.04; Dice: 97.78 + 2.47; IoU: 95.76 + 4.18), it demonstrates significantly enhanced generalization capabilities on out-of-distribution data, surpassing all U-Net variants by margins of up to 23 percent.