Combining HoloLens with Instant-NeRFs: Advanced Real-Time 3D Mobile Mapping
Dennis Haitz, Boris Jutzi, Markus Ulrich, Miriam Jaeger, Patrick, Huebner

TL;DR
This paper presents a real-time 3D reconstruction method using HoloLens 2 and Instant-NeRFs, enabling fast, radiometrically detailed point cloud extraction in mobile mapping scenarios.
Contribution
It introduces a novel integration of HoloLens with Instant-NeRFs for real-time 3D scene reconstruction on mobile devices, achieving unprecedented speed and detail.
Findings
Extracts 5 million scene points within 1 second
Outperforms traditional grid point sampling methods by orders of magnitude
Enables real-time, radiometrically detailed 3D mapping on mobile platforms
Abstract
This work represents a large step into modern ways of fast 3D reconstruction based on RGB camera images. Utilizing a Microsoft HoloLens 2 as a multisensor platform that includes an RGB camera and an inertial measurement unit for SLAM-based camera-pose determination, we train a Neural Radiance Field (NeRF) as a neural scene representation in real-time with the acquired data from the HoloLens. The HoloLens is connected via Wifi to a high-performance PC that is responsible for the training and 3D reconstruction. After the data stream ends, the training is stopped and the 3D reconstruction is initiated, which extracts a point cloud of the scene. With our specialized inference algorithm, five million scene points can be extracted within 1 second. In addition, the point cloud also includes radiometry per point. Our method of 3D reconstruction outperforms grid point sampling with NeRFs by…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsRobotics and Sensor-Based Localization · Advanced Vision and Imaging · Optical measurement and interference techniques
