Segmentation of kidney stones in endoscopic video feeds

TL;DR

This study develops and evaluates deep learning models for real-time segmentation of kidney stones in endoscopic videos, achieving over 90% accuracy and 30 fps, supporting clinical automation.

Contribution

It introduces a pipeline for dataset creation and adapts three deep learning models for kidney stone segmentation in endoscopic videos, demonstrating real-time performance.

Findings

Achieved over 90% segmentation accuracy.

Model runs at 30 frames per second.

Supports potential for clinical automation.

Abstract

Image segmentation has been increasingly applied in medical settings as recent developments have skyrocketed the potential applications of deep learning. Urology, specifically, is one field of medicine that is primed for the adoption of a real-time image segmentation system with the long-term aim of automating endoscopic stone treatment. In this project, we explored supervised deep learning models to annotate kidney stones in surgical endoscopic video feeds. In this paper, we describe how we built a dataset from the raw videos and how we developed a pipeline to automate as much of the process as possible. For the segmentation task, we adapted and analyzed three baseline deep learning models -- U-Net, U-Net++, and DenseNet -- to predict annotations on the frames of the endoscopic videos with the highest accuracy above 90\%. To show clinical potential for real-time use, we also confirmed that our best trained model can accurately annotate new videos at 30 frames per second. Our results demonstrate that the proposed method justifies continued development and study of image segmentation to annotate ureteroscopic video feeds.