Budget-Aware Uncertainty for Radiotherapy Segmentation QA Using nnU-Net

TL;DR

This paper introduces a budget-aware uncertainty-driven QA framework for radiotherapy segmentation using nnU-Net, enhancing calibration and uncertainty estimation to guide manual review efficiently.

Contribution

It proposes combining uncertainty quantification and post-hoc calibration methods to improve QA in radiotherapy segmentation, especially under realistic revision constraints.

Findings

Temperature scaling significantly improves calibration.

Calibrated checkpoint ensembles enhance uncertainty-error alignment.

Uncertainty maps effectively highlight regions needing manual edits.

Abstract

Accurate delineation of the Clinical Target Volume (CTV) is essential for radiotherapy planning, yet remains time-consuming and difficult to assess, especially for complex treatments such as Total Marrow and Lymph Node Irradiation (TMLI). While deep learning-based auto-segmentation can reduce workload, safe clinical deployment requires reliable cues indicating where models may be wrong. In this work, we propose a budget-aware uncertainty-driven quality assurance (QA) framework built on nnU-Net, combining uncertainty quantification and post-hoc calibration to produce voxel-wise uncertainty maps (based on predictive entropy) that can guide targeted manual review. We compare temperature scaling (TS), deep ensembles (DE), checkpoint ensembles (CE), and test-time augmentation (TTA), evaluated both individually and in combination on TMLI as a representative use case. Reliability is assessed through ROI-masked calibration metrics and uncertainty--error alignment under realistic revision constraints, summarized as AUC over the top 0-5% most uncertain voxels. Across configurations, segmentation accuracy remains stable, whereas TS substantially improves calibration. Uncertainty-error alignment improves most with calibrated checkpoint-based inference, leading to uncertainty maps that highlight more consistently regions requiring manual edits. Overall, integrating calibration with efficient ensembling seems a promising strategy to implement a budget-aware QA workflow for radiotherapy segmentation.