Systematic Clinical Evaluation of A Deep Learning Method for Medical Image Segmentation: Radiosurgery Application

TL;DR

This paper systematically evaluates a deep learning method for 3D medical image segmentation in radiosurgery, demonstrating improved agreement among raters, reduced detection disagreements, and faster delineation, with detailed analysis and clinical relevance.

Contribution

It provides a comprehensive clinical evaluation of a DL segmentation method, including detailed analysis and practical insights for medical image segmentation.

Findings

Reduces detection disagreement ratio from 0.162 to 0.085

Improves inter-rater surface Dice Score from 0.845 to 0.871

Accelerates delineation process by 1.6 to 2.0 times

Abstract

We systematically evaluate a Deep Learning (DL) method in a 3D medical image segmentation task. Our segmentation method is integrated into the radiosurgery treatment process and directly impacts the clinical workflow. With our method, we address the relative drawbacks of manual segmentation: high inter-rater contouring variability and high time consumption of the contouring process. The main extension over the existing evaluations is the careful and detailed analysis that could be further generalized on other medical image segmentation tasks. Firstly, we analyze the changes in the inter-rater detection agreement. We show that the segmentation model reduces the ratio of detection disagreements from 0.162 to 0.085 (p < 0.05). Secondly, we show that the model improves the inter-rater contouring agreement from 0.845 to 0.871 surface Dice Score (p < 0.05). Thirdly, we show that the model accelerates the delineation process in between 1.6 and 2.0 times (p < 0.05). Finally, we design the setup of the clinical experiment to either exclude or estimate the evaluation biases, thus preserve the significance of the results. Besides the clinical evaluation, we also summarize the intuitions and practical ideas for building an efficient DL-based model for 3D medical image segmentation.