Ground Awareness in Deep Learning for Large Outdoor Point Cloud Segmentation

TL;DR

This study demonstrates that incorporating relative elevation data, derived from Digital Terrain Models, significantly improves the accuracy of semantic segmentation in large outdoor point clouds using deep learning.

Contribution

It introduces the use of relative elevation features in deep learning models for outdoor point cloud segmentation, showing consistent performance gains across diverse datasets.

Findings

Performance improved by up to 3.7 percentage points in F1 score.

Relative elevation features enhance long-range dependency modeling.

Additional local features showed variable effectiveness.

Abstract

This paper presents an analysis of utilizing elevation data to aid outdoor point cloud semantic segmentation through existing machine-learning networks in remote sensing, specifically in urban, built-up areas. In dense outdoor point clouds, the receptive field of a machine learning model may be too small to accurately determine the surroundings and context of a point. By computing Digital Terrain Models (DTMs) from the point clouds, we extract the relative elevation feature, which is the vertical distance from the terrain to a point. RandLA-Net is employed for efficient semantic segmentation of large-scale point clouds. We assess its performance across three diverse outdoor datasets captured with varying sensor technologies and sensor locations. Integration of relative elevation data leads to consistent performance improvements across all three datasets, most notably in the Hessigheim dataset, with an increase of 3.7 percentage points in average F1 score from 72.35% to 76.01%, by establishing long-range dependencies between ground and objects. We also explore additional local features such as planarity, normal vectors, and 2D features, but their efficacy varied based on the characteristics of the point cloud. Ultimately, this study underscores the important role of the non-local relative elevation feature for semantic segmentation of point clouds in remote sensing applications.