Semantic 3D Occupancy Mapping through Efficient High Order CRFs

TL;DR

This paper introduces a real-time semantic 3D mapping system that combines CNN segmentation with an efficient high-order CRF model to improve accuracy in large-scale environments.

Contribution

It presents a novel incremental mapping system using a 3D occupancy grid and a high-order CRF for label optimization, enabling accurate large-scale semantic mapping.

Findings

Improved segmentation accuracy by 10% on KITTI dataset.

Developed an efficient mean field inference for high-order CRFs.

Built a memory-efficient 3D scrolling occupancy grid.

Abstract

Semantic 3D mapping can be used for many applications such as robot navigation and virtual interaction. In recent years, there has been great progress in semantic segmentation and geometric 3D mapping. However, it is still challenging to combine these two tasks for accurate and large-scale semantic mapping from images. In the paper, we propose an incremental and (near) real-time semantic mapping system. A 3D scrolling occupancy grid map is built to represent the world, which is memory and computationally efficient and bounded for large scale environments. We utilize the CNN segmentation as prior prediction and further optimize 3D grid labels through a novel CRF model. Superpixels are utilized to enforce smoothness and form robust P N high order potential. An efficient mean field inference is developed for the graph optimization. We evaluate our system on the KITTI dataset and improve the segmentation accuracy by 10% over existing systems.