De-noising, Stabilizing and Completing 3D Reconstructions On-the-go using Plane Priors

TL;DR

This paper introduces a fast, robust method for enhancing online 3D reconstructions by leveraging planar surface priors to denoise, stabilize, and complete maps in real-time on mobile devices, improving accuracy and semantic understanding.

Contribution

The paper presents a novel approach that integrates plane priors into real-time 3D reconstruction, significantly improving surface quality and scene understanding on mobile platforms.

Findings

Effective noise reduction and surface stabilization in real-time reconstructions.

Successful filling of occluded and unobserved regions using plane priors.

Semantic segmentation benefits from plane fitting in complex environments.

Abstract

Creating 3D maps on robots and other mobile devices has become a reality in recent years. Online 3D reconstruction enables many exciting applications in robotics and AR/VR gaming. However, the reconstructions are noisy and generally incomplete. Moreover, during onine reconstruction, the surface changes with every newly integrated depth image which poses a significant challenge for physics engines and path planning algorithms. This paper presents a novel, fast and robust method for obtaining and using information about planar surfaces, such as walls, floors, and ceilings as a stage in 3D reconstruction based on Signed Distance Fields. Our algorithm recovers clean and accurate surfaces, reduces the movement of individual mesh vertices caused by noise during online reconstruction and fills in the occluded and unobserved regions. We implemented and evaluated two different strategies to generate plane candidates and two strategies for merging them. Our implementation is optimized to run in real-time on mobile devices such as the Tango tablet. In an extensive set of experiments, we validated that our approach works well in a large number of natural environments despite the presence of significant amount of occlusion, clutter and noise, which occur frequently. We further show that plane fitting enables in many cases a meaningful semantic segmentation of real-world scenes.