Campus3D: A Photogrammetry Point Cloud Benchmark for Hierarchical Understanding of Outdoor Scene

TL;DR

This paper introduces Campus3D, a richly-annotated 3D point cloud dataset from UAV images of NUS campus, along with a hierarchical segmentation framework and benchmark results for outdoor scene understanding.

Contribution

It provides a new hierarchical 3D point cloud dataset and a novel multi-task learning framework with hierarchical ensemble for outdoor scene segmentation.

Findings

Proposed method outperforms existing approaches in hierarchical segmentation.

Hierarchical annotations improve segmentation accuracy.

Benchmark results are publicly available online.

Abstract

Learning on 3D scene-based point cloud has received extensive attention as its promising application in many fields, and well-annotated and multisource datasets can catalyze the development of those data-driven approaches. To facilitate the research of this area, we present a richly-annotated 3D point cloud dataset for multiple outdoor scene understanding tasks and also an effective learning framework for its hierarchical segmentation task. The dataset was generated via the photogrammetric processing on unmanned aerial vehicle (UAV) images of the National University of Singapore (NUS) campus, and has been point-wisely annotated with both hierarchical and instance-based labels. Based on it, we formulate a hierarchical learning problem for 3D point cloud segmentation and propose a measurement evaluating consistency across various hierarchies. To solve this problem, a two-stage method including multi-task (MT) learning and hierarchical ensemble (HE) with consistency consideration is proposed. Experimental results demonstrate the superiority of the proposed method and potential advantages of our hierarchical annotations. In addition, we benchmark results of semantic and instance segmentation, which is accessible online at https://3d.dataset.site with the dataset and all source codes.