Towards PerSense++: Advancing Training-Free Personalized Instance Segmentation in Dense Images

TL;DR

PerSense++ is a training-free, model-agnostic framework for personalized instance segmentation in dense images, introducing novel modules and a new benchmark to improve segmentation accuracy in complex scenes.

Contribution

We propose PerSense++, an advanced training-free framework with novel modules and a new benchmark, significantly improving dense scene segmentation performance.

Findings

PerSense++ outperforms existing methods in dense image segmentation.

The proposed modules enhance robustness and accuracy in cluttered scenes.

PerSense-D benchmark facilitates evaluation of personalized segmentation methods.

Abstract

Segmentation in dense visual scenes poses significant challenges due to occlusions, background clutter, and scale variations. To address this, we introduce PerSense, an end-to-end, training-free, and model-agnostic one-shot framework for Personalized instance Segmentation in dense images. PerSense employs a novel Instance Detection Module (IDM) that leverages density maps (DMs) to generate instance-level candidate point prompts, followed by a Point Prompt Selection Module (PPSM) that filters false positives via adaptive thresholding and spatial gating. A feedback mechanism further enhances segmentation by automatically selecting effective exemplars to improve DM quality. We additionally present PerSense++, an enhanced variant that incorporates three additional components to improve robustness in cluttered scenes: (i) a diversity-aware exemplar selection strategy that leverages feature and scale diversity for better DM generation; (ii) a hybrid IDM combining contour and peak-based prompt generation for improved instance separation within complex density patterns; and (iii) an Irrelevant Mask Rejection Module (IMRM) that discards spatially inconsistent masks using outlier analysis. Finally, to support this underexplored task, we introduce PerSense-D, a dedicated benchmark for personalized segmentation in dense images. Extensive experiments across multiple benchmarks demonstrate that PerSense++ outperforms existing methods in dense settings.