Impact of deep learning model uncertainty on manual corrections to auto-segmentation in prostate cancer radiotherapy

TL;DR

This study investigates how deep learning uncertainty maps influence radiation oncologists' manual correction of prostate segmentation, showing that uncertainty information can improve efficiency and trust in AI-assisted radiotherapy planning.

Contribution

It demonstrates that visualized uncertainty maps affect clinicians' editing behavior, confidence, and time efficiency in prostate radiotherapy segmentation.

Findings

Uncertainty maps lead to fewer edits in low-uncertainty regions.

Segmentation time decreased by 1-2 minutes with uncertainty maps.

Clinicians' confidence and perception of quality varied with uncertainty information.

Abstract

Background: Deep learning (DL)-based organ segmentation is increasingly used in radiotherapy, yet voxel-wise DL uncertainty maps are rarely presented to clinicians. Purpose: This study assessed how DL-generated uncertainty maps impact radiation oncologists during manual correction of prostate radiotherapy DL segmentations. Methods: Two nnUNet models were trained by 10-fold cross-validation on 434 MRI-only prostate cancer cases to segment the prostate and rectum. Each model was evaluated on 35 independent cases. Voxel-wise uncertainty was calculated using the SoftMax standard deviation (n=10) and visualized as a color-coded map. Four oncologists performed segmentation in two steps: Step 1: Rated segmentation quality and confidence using Likert scales and edited DL segmentations without uncertainty maps. Step 2 ($\geq 4$ weeks later): Repeated step 1, but with uncertainty maps available. Segmentation time was recorded for both steps, and oncologists provided qualitative free-text feedback. Histogram analysis compared voxel edits across uncertainty levels. Results: DL segmentations showed high agreement with oncologist edits. Quality ratings varied: rectum segmentation ratings slightly decreased overall in step 2, while prostate ratings differed among oncologists. Confidence ratings also varied. Three oncologists reduced segmentation time with uncertainty maps, saving 1-2 minutes per case. Histogram analysis showed 50% fewer edits for step 2 in low-uncertainty areas. Conclusions: Presenting DL segmentation uncertainty information to radiation oncologists influences their decision-making, quality perception, and confidence in the DL segmentations. Low-uncertainty regions were edited less frequently, indicating increased trust in DL predictions. Uncertainty maps improve efficiency by reducing segmentation time and can be a valuable clinical tool, enhancing radiotherapy planning efficiency.