Label tree semantic losses for rich multi-class medical image segmentation

TL;DR

This paper introduces hierarchical, tree-based semantic loss functions for medical image segmentation, improving accuracy by exploiting label inter-class relationships, especially with complex label sets.

Contribution

It proposes novel hierarchical loss functions that leverage label structure and extends them to sparse annotation scenarios, enhancing segmentation performance.

Findings

Consistent improvements over baseline methods in two medical imaging tasks.

Wasserstein-based loss performs best in whole-brain parcellation.

Hierarchy-weighted supervision excels with sparse annotations.

Abstract

Rich and accurate medical image segmentation is poised to underpin the next generation of AI-defined clinical practice by delineating critical anatomy for pre-operative planning, guiding real-time intra-operative navigation, and supporting precise post-operative assessment. However, commonly used learning methods for medical and surgical imaging segmentation tasks penalise all errors equivalently and thus fail to exploit any inter-class semantics in the label space. This becomes particularly problematic as the cardinality and richness of labels increases to include subtly different classes. In this work, we propose two tree-based semantic loss functions which take advantage of a hierarchical organisation of the labels. We further incorporate our losses in a recently proposed approach for training with sparse, background-free annotations to extend the applicability of our proposed losses. Extensive experiments are reported on two medical and surgical imaging segmentation tasks, namely head MRI for whole brain parcellation with full supervision and neurosurgical hyperspectral imaging for scene understanding with sparse annotations. Results demonstrate consistent improvements over the evaluated task-specific baselines, with the strongest support for the Wasserstein-based compound loss in whole-brain parcellation and for hierarchy-weighted top-level supervision in the sparse HSI setting.