Scale Disparity of Instances in Interactive Point Cloud Segmentation

TL;DR

This paper introduces ClickFormer, a novel interactive point cloud segmentation model that effectively handles scale disparity of instances, including both thing and stuff categories, outperforming existing methods in accuracy and efficiency.

Contribution

The paper proposes a new model with a query augmentation module and global attention mechanism to improve segmentation of scale-diverse instances in interactive point cloud segmentation.

Findings

Outperforms existing methods on indoor and outdoor datasets.

Achieves more accurate segmentation with fewer user clicks.

Effectively segments both thing and stuff categories across scales.

Abstract

Interactive point cloud segmentation has become a pivotal task for understanding 3D scenes, enabling users to guide segmentation models with simple interactions such as clicks, therefore significantly reducing the effort required to tailor models to diverse scenarios and new categories. However, in the realm of interactive segmentation, the meaning of instance diverges from that in instance segmentation, because users might desire to segment instances of both thing and stuff categories that vary greatly in scale. Existing methods have focused on thing categories, neglecting the segmentation of stuff categories and the difficulties arising from scale disparity. To bridge this gap, we propose ClickFormer, an innovative interactive point cloud segmentation model that accurately segments instances of both thing and stuff categories. We propose a query augmentation module to augment click queries by a global query sampling strategy, thus maintaining consistent performance across different instance scales. Additionally, we employ global attention in the query-voxel transformer to mitigate the risk of generating false positives, along with several other network structure improvements to further enhance the model's segmentation performance. Experiments demonstrate that ClickFormer outperforms existing interactive point cloud segmentation methods across both indoor and outdoor datasets, providing more accurate segmentation results with fewer user clicks in an open-world setting.