When Experts Disagree: Characterizing Annotator Variability for Vessel Segmentation in DSA Images

TL;DR

This paper investigates the variability among expert annotations of cranial vessel segmentations in DSA images to quantify uncertainty and inform improved annotation strategies and automatic segmentation methods.

Contribution

It introduces a method to analyze and quantify segmentation uncertainty due to annotator variability in vessel segmentation tasks.

Findings

Quantified segmentation uncertainty among multiple annotators.

Discussed how uncertainty can guide additional annotations.

Proposed approaches for uncertainty-aware automatic segmentation.

Abstract

We analyze the variability among segmentations of cranial blood vessels in 2D DSA performed by multiple annotators in order to characterize and quantify segmentation uncertainty. We use this analysis to quantify segmentation uncertainty and discuss ways it can be used to guide additional annotations and to develop uncertainty-aware automatic segmentation methods.