PreSem-Surf: RGB-D Surface Reconstruction with Progressive Semantic Modeling and SG-MLP Pre-Rendering Mechanism

TL;DR

PreSem-Surf is an efficient RGB-D surface reconstruction method that integrates semantic information and novel neural network structures to produce high-quality 3D scenes quickly.

Contribution

It introduces a new SG-MLP sampling structure and progressive semantic modeling within the NeRF framework for improved scene reconstruction.

Findings

Achieves top performance in C-L1, F-score, and IoU metrics.

Reduces training time while enhancing scene understanding.

Demonstrates effectiveness on synthetic scenes.

Abstract

This paper proposes PreSem-Surf, an optimized method based on the Neural Radiance Field (NeRF) framework, capable of reconstructing high-quality scene surfaces from RGB-D sequences in a short time. The method integrates RGB, depth, and semantic information to improve reconstruction performance. Specifically, a novel SG-MLP sampling structure combined with PR-MLP (Preconditioning Multilayer Perceptron) is introduced for voxel pre-rendering, allowing the model to capture scene-related information earlier and better distinguish noise from local details. Furthermore, progressive semantic modeling is adopted to extract semantic information at increasing levels of precision, reducing training time while enhancing scene understanding. Experiments on seven synthetic scenes with six evaluation metrics show that PreSem-Surf achieves the best performance in C-L1, F-score, and IoU, while maintaining competitive results in NC, Accuracy, and Completeness, demonstrating its effectiveness and practical applicability.