Interactive segmentation using U-Net with weight map and dynamic user interactions

TL;DR

This paper introduces an interactive segmentation method using a U-Net model with dynamically adapted user clicks forming a weight map, significantly improving segmentation accuracy with minimal user input.

Contribution

The work presents a novel weighted loss function based on dynamic user clicks, enhancing interactive segmentation performance over standard U-Net models.

Findings

Accuracy improved by 5.60% on spleen images

Accuracy improved by 10.39% on colon cancer images

Effective with only a single user interaction

Abstract

Interactive segmentation has recently attracted attention for specialized tasks where expert input is required to further enhance the segmentation performance. In this work, we propose a novel interactive segmentation framework, where user clicks are dynamically adapted in size based on the current segmentation mask. The clicked regions form a weight map and are fed to a deep neural network as a novel weighted loss function. To evaluate our loss function, an interactive U-Net (IU-Net) model which applies both foreground and background user clicks as the main method of interaction is employed. We train and validate on the BCV dataset, while testing on spleen and colon cancer CT images from the MSD dataset to improve the overall segmentation accuracy in comparison to the standard U-Net using our weighted loss function. Applying dynamic user click sizes increases the overall accuracy by 5.60% and 10.39% respectively by utilizing only a single user interaction.