Every Component Counts: Rethinking the Measure of Success for Medical Semantic Segmentation in Multi-Instance Segmentation Tasks

TL;DR

This paper introduces CC-Metrics, a new evaluation protocol for medical semantic segmentation that assesses each connected component individually, reducing bias towards larger components and improving the clinical relevance of segmentation metrics.

Contribution

The paper proposes a novel per-component evaluation method for semantic segmentation metrics, aligning them better with clinical needs in multi-instance scenarios like tumor detection.

Findings

CC-Metrics reduces bias towards larger metastases in evaluation.

It improves the informativeness of distance-based metrics for small changes.

The approach aligns segmentation evaluation with clinical relevance.

Abstract

We present Connected-Component~(CC)-Metrics, a novel semantic segmentation evaluation protocol, targeted to align existing semantic segmentation metrics to a multi-instance detection scenario in which each connected component matters. We motivate this setup in the common medical scenario of semantic metastases segmentation in a full-body PET/CT. We show how existing semantic segmentation metrics suffer from a bias towards larger connected components contradicting the clinical assessment of scans in which tumor size and clinical relevance are uncorrelated. To rebalance existing segmentation metrics, we propose to evaluate them on a per-component basis thus giving each tumor the same weight irrespective of its size. To match predictions to ground-truth segments, we employ a proximity-based matching criterion, evaluating common metrics locally at the component of interest. Using this approach, we break free of biases introduced by large metastasis for overlap-based metrics such as Dice or Surface Dice. CC-Metrics also improves distance-based metrics such as Hausdorff Distances which are uninformative for small changes that do not influence the maximum or 95th percentile, and avoids pitfalls introduced by directly combining counting-based metrics with overlap-based metrics as it is done in Panoptic Quality.