Unleashing the Power of Prompt-driven Nucleus Instance Segmentation

TL;DR

This paper introduces a prompt-driven framework combining a nucleus prompter and SAM for accurate, post-processing-free nucleus instance segmentation in histology images, achieving state-of-the-art results.

Contribution

It presents a novel prompt-driven approach with a nucleus prompter and fine-tuned SAM, incorporating negative prompts to improve segmentation of overlapping nuclei.

Findings

Achieves new state-of-the-art performance on three benchmarks.

Eliminates the need for complex post-processing.

Effectively segments overlapping nuclei using negative prompts.

Abstract

Nucleus instance segmentation in histology images is crucial for a broad spectrum of clinical applications. Current dominant algorithms rely on regression of nuclear proxy maps. Distinguishing nucleus instances from the estimated maps requires carefully curated post-processing, which is error-prone and parameter-sensitive. Recently, the Segment Anything Model (SAM) has earned huge attention in medical image segmentation, owing to its impressive generalization ability and promptable property. Nevertheless, its potential on nucleus instance segmentation remains largely underexplored. In this paper, we present a novel prompt-driven framework that consists of a nucleus prompter and SAM for automatic nucleus instance segmentation. Specifically, the prompter learns to generate a unique point prompt for each nucleus while the SAM is fine-tuned to output the corresponding mask for the prompted nucleus. Furthermore, we propose the inclusion of adjacent nuclei as negative prompts to enhance the model's capability to identify overlapping nuclei. Without complicated post-processing, our proposed method sets a new state-of-the-art performance on three challenging benchmarks. Code is available at \url{github.com/windygoo/PromptNucSeg}