DS$^2$Net: Detail-Semantic Deep Supervision Network for Medical Image Segmentation

TL;DR

DS$^2$Net introduces a multi-view deep supervision approach combining detail and semantic features with an uncertainty-based loss, significantly improving medical image segmentation across various benchmarks.

Contribution

The paper proposes DS$^2$Net, a novel multi-view deep supervision network with detail and semantic modules and an uncertainty-based loss for enhanced medical image segmentation.

Findings

Outperforms state-of-the-art methods on six benchmarks

Effectively combines detail and semantic supervision

Demonstrates robustness across different medical imaging modalities

Abstract

Deep Supervision Networks exhibit significant efficacy for the medical imaging community. Nevertheless, existing work merely supervises either the coarse-grained semantic features or fine-grained detailed features in isolation, which compromises the fact that these two types of features hold vital relationships in medical image analysis. We advocate the powers of complementary feature supervision for medical image segmentation, by proposing a Detail-Semantic Deep Supervision Network (DS$^2$Net). DS$^2$Net navigates both low-level detailed and high-level semantic feature supervision through Detail Enhance Module (DEM) and Semantic Enhance Module (SEM). DEM and SEM respectively harness low-level and high-level feature maps to create detail and semantic masks for enhancing feature supervision. This is a novel shift from single-view deep supervision to multi-view deep supervision. DS$^2$Net is also equipped with a novel uncertainty-based supervision loss that adaptively assigns the supervision strength of features within distinct scales based on their uncertainty, thus circumventing the sub-optimal heuristic design that typifies previous works. Through extensive experiments on six benchmarks captured under either colonoscopy, ultrasound and microscope, we demonstrate that DS$^2$Net consistently outperforms state-of-the-art methods for medical image analysis.