PerSense: Training-Free Personalized Instance Segmentation in Dense Images

TL;DR

PerSense is a training-free, model-agnostic framework for personalized instance segmentation in dense images, utilizing novel modules for automatic point prompt generation and refinement, with an introduced benchmark for evaluation.

Contribution

We propose PerSense, a novel training-free framework with new modules for automatic prompt generation and refinement, and introduce PerSense-D benchmark for dense image segmentation.

Findings

PerSense outperforms state-of-the-art methods in dense image segmentation.

The framework effectively reduces false positives and improves accuracy.

PerSense-D provides a new standard for evaluating dense instance segmentation.

Abstract

The emergence of foundational models has significantly advanced segmentation approaches. However, challenges still remain in dense scenarios, where occlusions, scale variations, and clutter impede precise instance delineation. To address this, we propose PerSense, an end-to-end, training-free, and model-agnostic one-shot framework for Personalized instance Segmentation in dense images. We start with developing a new baseline capable of automatically generating instance-level point prompts via proposing a novel Instance Detection Module (IDM) that leverages density maps (DMs), encapsulating spatial distribution of objects in an image. To reduce false positives, we design the Point Prompt Selection Module (PPSM), which refines the output of IDM based on adaptive threshold and spatial gating. Both IDM and PPSM seamlessly integrate into our model-agnostic framework. Furthermore, we introduce a feedback mechanism that enables PerSense to improve the accuracy of DMs by automating the exemplar selection process for DM generation. Finally, to advance research in this relatively underexplored area, we introduce PerSense-D, an evaluation benchmark for instance segmentation in dense images. Our extensive experiments establish PerSense's superiority over SOTA in dense settings.