SUM: A Benchmark Dataset of Semantic Urban Meshes

TL;DR

This paper introduces a new benchmark dataset of semantic urban meshes, an annotation framework, and an open-source tool to facilitate 3D urban scene understanding and semantic segmentation research.

Contribution

It provides a large, annotated urban mesh dataset, a semi-automatic annotation framework, and an open-source tool, advancing 3D semantic segmentation in urban environments.

Findings

Benchmark dataset covers 4 km2 in Helsinki

State-of-the-art segmentation methods evaluated on the dataset

Annotation framework saves approximately 600 hours of manual labeling

Abstract

Recent developments in data acquisition technology allow us to collect 3D texture meshes quickly. Those can help us understand and analyse the urban environment, and as a consequence are useful for several applications like spatial analysis and urban planning. Semantic segmentation of texture meshes through deep learning methods can enhance this understanding, but it requires a lot of labelled data. The contributions of this work are threefold: (1) a new benchmark dataset of semantic urban meshes, (2) a novel semi-automatic annotation framework, and (3) an annotation tool for 3D meshes. In particular, our dataset covers about 4 km2 in Helsinki (Finland), with six classes, and we estimate that we save about 600 hours of labelling work using our annotation framework, which includes initial segmentation and interactive refinement. We also compare the performance of several state-of-theart 3D semantic segmentation methods on the new benchmark dataset. Other researchers can use our results to train their networks: the dataset is publicly available, and the annotation tool is released as open-source.