Focused Specific Objects NeRF

TL;DR

This paper introduces a scene-specific NeRF approach utilizing semantic priors for faster training and improved rendering, along with weak supervision and novel scene editing techniques, applicable to all NeRF models.

Contribution

It proposes a semantic prior-based focus mechanism for NeRF, significantly accelerating training and enhancing rendering quality, with extensions to scene editing and self-supervised correction.

Findings

Training speed increased by 7.78 times

Applicable to all NeRF-based models

Enhanced scene editing with semantic masking

Abstract

Most NeRF-based models are designed for learning the entire scene, and complex scenes can lead to longer learning times and poorer rendering effects. This paper utilizes scene semantic priors to make improvements in fast training, allowing the network to focus on the specific targets and not be affected by complex backgrounds. The training speed can be increased by 7.78 times with better rendering effect, and small to medium sized targets can be rendered faster. In addition, this improvement applies to all NeRF-based models. Considering the inherent multi-view consistency and smoothness of NeRF, this paper also studies weak supervision by sparsely sampling negative ray samples. With this method, training can be further accelerated and rendering quality can be maintained. Finally, this paper extends pixel semantic and color rendering formulas and proposes a new scene editing technique that can achieve unique displays of the specific semantic targets or masking them in rendering. To address the problem of unsupervised regions incorrect inferences in the scene, we also designed a self-supervised loop that combines morphological operations and clustering.