FlyNeRF: NeRF-Based Aerial Mapping for High-Quality 3D Scene Reconstruction

TL;DR

FlyNeRF combines drone-based image capture with neural radiance fields and an AI-driven quality assessment to produce high-fidelity 3D environmental reconstructions, improving detail and accuracy.

Contribution

This paper introduces FlyNeRF, a novel system integrating NeRF with UAV data collection and an AI-based quality evaluation for enhanced 3D mapping.

Findings

Neural network for render quality assessment achieves 97% accuracy.

Adaptive image capture improves PSNR by 2.5 dB on average.

System demonstrates promising results for environmental monitoring and digital twins.

Abstract

Current methods for 3D reconstruction and environmental mapping frequently face challenges in achieving high precision, highlighting the need for practical and effective solutions. In response to this issue, our study introduces FlyNeRF, a system integrating Neural Radiance Fields (NeRF) with drone-based data acquisition for high-quality 3D reconstruction. Utilizing unmanned aerial vehicle (UAV) for capturing images and corresponding spatial coordinates, the obtained data is subsequently used for the initial NeRF-based 3D reconstruction of the environment. Further evaluation of the reconstruction render quality is accomplished by the image evaluation neural network developed within the scope of our system. According to the results of the image evaluation module, an autonomous algorithm determines the position for additional image capture, thereby improving the reconstruction quality. The neural network introduced for render quality assessment demonstrates an accuracy of 97%. Furthermore, our adaptive methodology enhances the overall reconstruction quality, resulting in an average improvement of 2.5 dB in Peak Signal-to-Noise Ratio (PSNR) for the 10% quantile. The FlyNeRF demonstrates promising results, offering advancements in such fields as environmental monitoring, surveillance, and digital twins, where high-fidelity 3D reconstructions are crucial.