Adapting Segment Anything Model for Unseen Object Instance Segmentation

TL;DR

This paper introduces UOIS-SAM, a data-efficient method that adapts the Segment Anything Model for unseen object instance segmentation, achieving state-of-the-art results with minimal training data in unstructured environments.

Contribution

The paper presents UOIS-SAM, combining a heatmap-based prompt generator and a hierarchical discrimination network to enhance SAM's performance on unseen object segmentation tasks.

Findings

Achieves state-of-the-art performance using only 10% of training data.

Effective in complex scenes with occlusion and texture-rich objects.

Robust across diverse datasets including OCID, OSD, PhoCAL, and HouseCat6D.

Abstract

Unseen Object Instance Segmentation (UOIS) is crucial for autonomous robots operating in unstructured environments. Previous approaches require full supervision on large-scale tabletop datasets for effective pretraining. In this paper, we propose UOIS-SAM, a data-efficient solution for the UOIS task that leverages SAM's high accuracy and strong generalization capabilities. UOIS-SAM integrates two key components: (i) a Heatmap-based Prompt Generator (HPG) to generate class-agnostic point prompts with precise foreground prediction, and (ii) a Hierarchical Discrimination Network (HDNet) that adapts SAM's mask decoder, mitigating issues introduced by the SAM baseline, such as background confusion and over-segmentation, especially in scenarios involving occlusion and texture-rich objects. Extensive experimental results on OCID, OSD, and additional photometrically challenging datasets including PhoCAL and HouseCat6D, demonstrate that, even using only 10% of the training samples compared to previous methods, UOIS-SAM achieves state-of-the-art performance in unseen object segmentation, highlighting its effectiveness and robustness in various tabletop scenes.