PLUG: Revisiting Amodal Segmentation with Foundation Model and Hierarchical Focus

TL;DR

PLUG introduces a novel amodal segmentation method leveraging foundation model priors and a hierarchical focus framework, significantly improving performance on key datasets despite limited annotations.

Contribution

The paper presents the first SAM-based amodal segmentation approach with a hierarchical focus framework and uncertainty-guided point loss, enhancing segmentation accuracy and efficiency.

Findings

Outperforms existing methods with large margins.

Achieves superior results with fewer parameters.

Effectively handles occlusion and ambiguous regions.

Abstract

Aiming to predict the complete shapes of partially occluded objects, amodal segmentation is an important step towards visual intelligence. With crucial significance, practical prior knowledge derives from sufficient training, while limited amodal annotations pose challenges to achieve better performance. To tackle this problem, utilizing the mighty priors accumulated in the foundation model, we propose the first SAM-based amodal segmentation approach, PLUG. Methodologically, a novel framework with hierarchical focus is presented to better adapt the task characteristics and unleash the potential capabilities of SAM. In the region level, due to the association and division in visible and occluded areas, inmodal and amodal regions are assigned as the focuses of distinct branches to avoid mutual disturbance. In the point level, we introduce the concept of uncertainty to explicitly assist the model in identifying and focusing on ambiguous points. Guided by the uncertainty map, a computation-economic point loss is applied to improve the accuracy of predicted boundaries. Experiments are conducted on several prominent datasets, and the results show that our proposed method outperforms existing methods with large margins. Even with fewer total parameters, our method still exhibits remarkable advantages.