JSENet: Joint Semantic Segmentation and Edge Detection Network for 3D Point Clouds

TL;DR

This paper introduces JSENet, a novel joint network for 3D semantic segmentation and edge detection, improving boundary accuracy and achieving state-of-the-art results on major datasets.

Contribution

The paper presents the first joint learning framework for 3D semantic edge detection and segmentation, with a new refinement module and loss function enhancing boundary quality.

Findings

Achieves comparable or superior segmentation performance to state-of-the-art methods.

Outperforms baseline methods in semantic edge detection.

Demonstrates effectiveness on S3DIS and ScanNet datasets.

Abstract

Semantic segmentation and semantic edge detection can be seen as two dual problems with close relationships in computer vision. Despite the fast evolution of learning-based 3D semantic segmentation methods, little attention has been drawn to the learning of 3D semantic edge detectors, even less to a joint learning method for the two tasks. In this paper, we tackle the 3D semantic edge detection task for the first time and present a new two-stream fully-convolutional network that jointly performs the two tasks. In particular, we design a joint refinement module that explicitly wires region information and edge information to improve the performances of both tasks. Further, we propose a novel loss function that encourages the network to produce semantic segmentation results with better boundaries. Extensive evaluations on S3DIS and ScanNet datasets show that our method achieves on par or better performance than the state-of-the-art methods for semantic segmentation and outperforms the baseline methods for semantic edge detection. Code release: https://github.com/hzykent/JSENet