Transforming the Interactive Segmentation for Medical Imaging

TL;DR

This paper introduces a Transformer-based interactive segmentation method for medical imaging that improves the refinement of challenging structures, supporting multi-category editing and outperforming existing methods on multiple datasets.

Contribution

A novel Transformer architecture for interactive segmentation that handles multiple categories and improves refinement of difficult structures in medical images.

Findings

Outperforms state-of-the-art methods on three datasets

Supports multi-category mask editing

Effective in segmenting challenging structures like cancer and small organs

Abstract

The goal of this paper is to interactively refine the automatic segmentation on challenging structures that fall behind human performance, either due to the scarcity of available annotations or the difficulty nature of the problem itself, for example, on segmenting cancer or small organs. Specifically, we propose a novel Transformer-based architecture for Interactive Segmentation (TIS), that treats the refinement task as a procedure for grouping pixels with similar features to those clicks given by the end users. Our proposed architecture is composed of Transformer Decoder variants, which naturally fulfills feature comparison with the attention mechanisms. In contrast to existing approaches, our proposed TIS is not limited to binary segmentations, and allows the user to edit masks for arbitrary number of categories. To validate the proposed approach, we conduct extensive experiments on three challenging datasets and demonstrate superior performance over the existing state-of-the-art methods. The project page is: https://wtliu7.github.io/tis/.