Pitfalls of topology-aware image segmentation

TL;DR

This paper critically examines the evaluation practices of topology-aware image segmentation methods in medical imaging, revealing significant pitfalls and proposing standards for fair assessment to improve real-world applicability.

Contribution

It identifies key flaws in benchmarking practices for topology-aware segmentation and offers recommendations to establish more reliable evaluation standards.

Findings

Inadequate connectivity choices affect evaluation outcomes.

Overlooked topological artifacts distort ground truth assessments.

Current metrics may misrepresent segmentation quality.

Abstract

Topological correctness, i.e., the preservation of structural integrity and specific characteristics of shape, is a fundamental requirement for medical imaging tasks, such as neuron or vessel segmentation. Despite the recent surge in topology-aware methods addressing this challenge, their real-world applicability is hindered by flawed benchmarking practices. In this paper, we identify critical pitfalls in model evaluation that include inadequate connectivity choices, overlooked topological artifacts in ground truth annotations, and inappropriate use of evaluation metrics. Through detailed empirical analysis, we uncover these issues' profound impact on the evaluation and ranking of segmentation methods. Drawing from our findings, we propose a set of actionable recommendations to establish fair and robust evaluation standards for topology-aware medical image segmentation methods.