GSTran: Joint Geometric and Semantic Coherence for Point Cloud Segmentation

TL;DR

GSTran introduces a transformer-based approach for point cloud segmentation that explicitly models local geometric disparities and global semantic dependencies, improving accuracy by preserving feature distinctiveness and capturing contextual information.

Contribution

The paper presents GSTran, a novel transformer network with local geometric and global semantic modules, addressing aggregation and dependency modeling issues in point cloud segmentation.

Findings

Outperforms existing methods on ShapeNetPart and S3DIS benchmarks.

Effectively amplifies geometric affinity and captures semantic context.

Demonstrates superior segmentation accuracy and robustness.

Abstract

Learning meaningful local and global information remains a challenge in point cloud segmentation tasks. When utilizing local information, prior studies indiscriminately aggregates neighbor information from different classes to update query points, potentially compromising the distinctive feature of query points. In parallel, inaccurate modeling of long-distance contextual dependencies when utilizing global information can also impact model performance. To address these issues, we propose GSTran, a novel transformer network tailored for the segmentation task. The proposed network mainly consists of two principal components: a local geometric transformer and a global semantic transformer. In the local geometric transformer module, we explicitly calculate the geometric disparity within the local region. This enables amplifying the affinity with geometrically similar neighbor points while suppressing the association with other neighbors. In the global semantic transformer module, we design a multi-head voting strategy. This strategy evaluates semantic similarity across the entire spatial range, facilitating the precise capture of contextual dependencies. Experiments on ShapeNetPart and S3DIS benchmarks demonstrate the effectiveness of the proposed method, showing its superiority over other algorithms. The code is available at https://github.com/LAB123-tech/GSTran.