Quality-Aware Memory Network for Interactive Volumetric Image Segmentation

TL;DR

This paper introduces a quality-aware memory network that enhances interactive 3D medical image segmentation by propagating user-guided initial segmentations across volumes and actively suggesting slices for refinement based on quality assessment.

Contribution

It presents a novel memory-augmented network with a quality assessment module for improved, interactive 3D medical image segmentation, enabling efficient propagation and active refinement.

Findings

Outperforms existing segmentation techniques on various datasets.

Effectively propagates segmentation information across slices.

Accurately estimates segmentation quality to guide user interactions.

Abstract

Despite recent progress of automatic medical image segmentation techniques, fully automatic results usually fail to meet the clinical use and typically require further refinement. In this work, we propose a quality-aware memory network for interactive segmentation of 3D medical images. Provided by user guidance on an arbitrary slice, an interaction network is firstly employed to obtain an initial 2D segmentation. The quality-aware memory network subsequently propagates the initial segmentation estimation bidirectionally over the entire volume. Subsequent refinement based on additional user guidance on other slices can be incorporated in the same manner. To further facilitate interactive segmentation, a quality assessment module is introduced to suggest the next slice to segment based on the current segmentation quality of each slice. The proposed network has two appealing characteristics: 1) The memory-augmented network offers the ability to quickly encode past segmentation information, which will be retrieved for the segmentation of other slices; 2) The quality assessment module enables the model to directly estimate the qualities of segmentation predictions, which allows an active learning paradigm where users preferentially label the lowest-quality slice for multi-round refinement. The proposed network leads to a robust interactive segmentation engine, which can generalize well to various types of user annotations (e.g., scribbles, boxes). Experimental results on various medical datasets demonstrate the superiority of our approach in comparison with existing techniques.