PointSeg: Real-Time Semantic Segmentation Based on 3D LiDAR Point Cloud

TL;DR

PointSeg is a fast, lightweight CNN-based method that performs real-time semantic segmentation of road objects from spherical images derived from 3D LiDAR data, suitable for autonomous driving.

Contribution

The paper introduces PointSeg, a novel real-time semantic segmentation approach using spherical images and a lightweight CNN based on SqueezeNet, optimized for mobile systems.

Findings

Achieves 90fps on a single GPU 1080ti

Maintains a good balance between accuracy and memory usage

Provides competitive accuracy for autonomous driving applications

Abstract

In this paper, we propose PointSeg, a real-time end-to-end semantic segmentation method for road-objects based on spherical images. We take the spherical image, which is transformed from the 3D LiDAR point clouds, as input of the convolutional neural networks (CNNs) to predict the point-wise semantic map. To make PointSeg applicable on a mobile system, we build the model based on the light-weight network, SqueezeNet, with several improvements. It maintains a good balance between memory cost and prediction performance. Our model is trained on spherical images and label masks projected from the KITTI 3D object detection dataset. Experiments show that PointSeg can achieve competitive accuracy with 90fps on a single GPU 1080ti. which makes it quite compatible for autonomous driving applications.