Resource-efficient Automatic Refinement of Segmentations via Weak Supervision from Light Feedback

TL;DR

SCORE is a weakly supervised framework that refines medical image segmentations using minimal feedback, reducing annotation effort while achieving high accuracy comparable to fully supervised methods.

Contribution

The paper introduces SCORE, a novel weakly supervised segmentation refinement method that requires only light feedback, unlike existing approaches that need extensive annotations.

Findings

Significantly improves initial segmentation predictions from TotalSegmentator.

Achieves performance comparable to fully supervised refinement methods.

Reduces supervision and annotation time in medical image segmentation.

Abstract

Delineating anatomical regions is a key task in medical image analysis. Manual segmentation achieves high accuracy but is labor-intensive and prone to variability, thus prompting the development of automated approaches. Recently, a breadth of foundation models has enabled automated segmentations across diverse anatomies and imaging modalities, but these may not always meet the clinical accuracy standards. While segmentation refinement strategies can improve performance, current methods depend on heavy user interactions or require fully supervised segmentations for training. Here, we present SCORE (Segmentation COrrection from Regional Evaluations), a weakly supervised framework that learns to refine mask predictions only using light feedback during training. Specifically, instead of relying on dense training image annotations, SCORE introduces a novel loss that leverages region-wise quality scores and over/under-segmentation error labels. We demonstrate SCORE on humerus CT scans, where it considerably improves initial predictions from TotalSegmentator, and achieves performance on par with existing refinement methods, while greatly reducing their supervision requirements and annotation time. Our code is available at: https://gitlab.inria.fr/adelangl/SCORE.