# 3D Graph Embedding Learning with a Structure-aware Loss Function for Point Cloud Semantic Instance Segmentation

## TL;DR

This paper presents a novel 3D point cloud segmentation method combining a structure-aware loss function, attention-based KNN, and graph convolutional networks to improve semantic and instance segmentation accuracy.

## Contribution

It introduces a structure-aware loss and attention-based KNN within an end-to-end trainable framework for enhanced 3D point cloud segmentation.

## Key findings

- Outperforms state-of-the-art on ScanNet benchmark
- Achieves superior results on NYUv2 dataset
- Provides both semantic and instance segmentation outputs

## Abstract

This paper introduces a novel approach for 3D semantic instance segmentation on point clouds. A 3D convolutional neural network called submanifold sparse convolutional network is used to generate semantic predictions and instance embeddings simultaneously. To obtain discriminative embeddings for each 3D instance, a structure-aware loss function is proposed which considers both the structure information and the embedding information. To get more consistent embeddings for each 3D instance, attention-based k nearest neighbour (KNN) is proposed to assign different weights for different neighbours. Based on the attention-based KNN, we add a graph convolutional network after the sparse convolutional network to get refined embeddings. Our network can be trained end-to-end. A simple mean-shift algorithm is utilized to cluster refined embeddings to get final instance predictions. As a result, our framework can output both the semantic prediction and the instance prediction. Experiments show that our approach outperforms all state-of-art methods on ScanNet benchmark and NYUv2 dataset.