FAWN: Floor-And-Walls Normal Regularization for Direct Neural TSDF Reconstruction

TL;DR

FAWN introduces a scene-structure-aware regularization for TSDF-based 3D reconstruction, improving accuracy by leveraging wall and floor orientations without additional inference constraints.

Contribution

It proposes a novel normal regularization method that incorporates scene semantics into TSDF reconstruction, enhancing existing methods' performance.

Findings

Improved reconstruction quality on multiple benchmarks.

More effective use of semantic information than existing approaches.

Compatible with various trainable TSDF pipelines.

Abstract

Leveraging 3D semantics for direct 3D reconstruction has a great potential yet unleashed. For instance, by assuming that walls are vertical, and a floor is planar and horizontal, we can correct distorted room shapes and eliminate local artifacts such as holes, pits, and hills. In this paper, we propose FAWN, a modification of truncated signed distance function (TSDF) reconstruction methods, which considers scene structure by detecting walls and floor in a scene, and penalizing the corresponding surface normals for deviating from the horizontal and vertical directions. Implemented as a 3D sparse convolutional module, FAWN can be incorporated into any trainable pipeline that predicts TSDF. Since FAWN requires 3D semantics only for training, no additional limitations on further use are imposed. We demonstrate, that FAWN-modified methods use semantics more effectively, than existing semantic-based approaches. Besides, we apply our modification to state-of-the-art TSDF reconstruction methods, and demonstrate a quality gain in SCANNET, ICL-NUIM, TUM RGB-D, and 7SCENES benchmarks.