BoxCell: Leveraging SAM for Cell Segmentation with Box Supervision

TL;DR

BoxCell introduces a novel weakly supervised cell segmentation framework that leverages the pre-trained SAM model with bounding box prompts, combining multiple masks through integer programming to significantly improve segmentation accuracy.

Contribution

This work presents a new method that uses SAM without fine-tuning for box-supervised cell segmentation, integrating multiple masks via integer programming.

Findings

Outperforms existing box supervised models by 6-10 Dice points

Effective use of SAM's pre-trained capabilities without fine-tuning

Achieves state-of-the-art results on three public datasets

Abstract

Cell segmentation in histopathological images is vital for diagnosis, and treatment of several diseases. Annotating data is tedious, and requires medical expertise, making it difficult to employ supervised learning. Instead, we study a weakly supervised setting, where only bounding box supervision is available, and present the use of Segment Anything (SAM) for this without any finetuning, i.e., directly utilizing the pre-trained model. We propose BoxCell, a cell segmentation framework that utilizes SAM's capability to interpret bounding boxes as prompts, \emph{both} at train and test times. At train time, gold bounding boxes given to SAM produce (pseudo-)masks, which are used to train a standalone segmenter. At test time, BoxCell generates two segmentation masks: (1) generated by this standalone segmenter, and (2) a trained object detector outputs bounding boxes, which are given as prompts to SAM to produce another mask. Recognizing complementary strengths, we reconcile the two segmentation masks using a novel integer programming formulation with intensity and spatial constraints. We experiment on three publicly available cell segmentation datasets namely, CoNSep, MoNuSeg, and TNBC, and find that BoxCell significantly outperforms existing box supervised image segmentation models, obtaining 6-10 point Dice gains.