A Proposal for Semantic Map Representation and Evaluation

TL;DR

This paper proposes a standardized, extensible formalism for semantic map representation and introduces a benchmarking dataset and evaluation procedures to advance research in semantic mapping.

Contribution

It introduces a uniform formalism for semantic map representation and provides a dataset and evaluation framework for benchmarking semantic mapping methods.

Findings

A formalism for semantic map representation is defined.

A dataset based on real sensor data is constructed for benchmarking.

Evaluation metrics and tools for semantic mapping are proposed.

Abstract

Semantic mapping is the incremental process of "mapping" relevant information of the world (i.e., spatial information, temporal events, agents and actions) to a formal description supported by a reasoning engine. Current research focuses on learning the semantic of environments based on their spatial location, geometry and appearance. Many methods to tackle this problem have been proposed, but the lack of a uniform representation, as well as standard benchmarking suites, prevents their direct comparison. In this paper, we propose a standardization in the representation of semantic maps, by defining an easily extensible formalism to be used on top of metric maps of the environments. Based on this, we describe the procedure to build a dataset (based on real sensor data) for benchmarking semantic mapping techniques, also hypothesizing some possible evaluation metrics. Nevertheless, by providing a tool for the construction of a semantic map ground truth, we aim at the contribution of the scientific community in acquiring data for populating the dataset.