SemAffiNet: Semantic-Affine Transformation for Point Cloud Segmentation

TL;DR

SemAffiNet introduces a semantic-affine transformation technique that enhances mid-level features with class-specific information, improving point cloud segmentation accuracy by capturing global context and reducing confusion between similar local parts.

Contribution

The paper proposes a novel semantic-affine transformation method and integrates it into a Transformer-based network for improved 3D point cloud segmentation.

Findings

Outperforms existing methods on ScanNetV2 and NYUv2 datasets.

Demonstrates superior generalization across different segmentation baselines.

Qualitative results show clearer category boundaries and reduced confusion.

Abstract

Conventional point cloud semantic segmentation methods usually employ an encoder-decoder architecture, where mid-level features are locally aggregated to extract geometric information. However, the over-reliance on these class-agnostic local geometric representations may raise confusion between local parts from different categories that are similar in appearance or spatially adjacent. To address this issue, we argue that mid-level features can be further enhanced with semantic information, and propose semantic-affine transformation that transforms features of mid-level points belonging to different categories with class-specific affine parameters. Based on this technique, we propose SemAffiNet for point cloud semantic segmentation, which utilizes the attention mechanism in the Transformer module to implicitly and explicitly capture global structural knowledge within local parts for overall comprehension of each category. We conduct extensive experiments on the ScanNetV2 and NYUv2 datasets, and evaluate semantic-affine transformation on various 3D point cloud and 2D image segmentation baselines, where both qualitative and quantitative results demonstrate the superiority and generalization ability of our proposed approach. Code is available at https://github.com/wangzy22/SemAffiNet.