NeRF-SLAM: Real-Time Dense Monocular SLAM with Neural Radiance Fields

TL;DR

This paper introduces NeRF-SLAM, a real-time monocular SLAM system that combines neural radiance fields with depth and pose estimation to produce accurate 3D scene reconstructions.

Contribution

It presents a novel pipeline integrating dense monocular SLAM with neural radiance fields, enabling real-time, accurate scene reconstruction from monocular images.

Findings

Achieves up to 179% better PSNR than competitors.

Attains 86% better L1 depth accuracy.

Operates in real-time using only monocular images.

Abstract

We propose a novel geometric and photometric 3D mapping pipeline for accurate and real-time scene reconstruction from monocular images. To achieve this, we leverage recent advances in dense monocular SLAM and real-time hierarchical volumetric neural radiance fields. Our insight is that dense monocular SLAM provides the right information to fit a neural radiance field of the scene in real-time, by providing accurate pose estimates and depth-maps with associated uncertainty. With our proposed uncertainty-based depth loss, we achieve not only good photometric accuracy, but also great geometric accuracy. In fact, our proposed pipeline achieves better geometric and photometric accuracy than competing approaches (up to 179% better PSNR and 86% better L1 depth), while working in real-time and using only monocular images.