Ternary-Type Opacity and Hybrid Odometry for RGB NeRF-SLAM

TL;DR

This paper introduces a novel ternary opacity model and hybrid odometry scheme to enhance RGB-only NeRF-SLAM, achieving state-of-the-art accuracy and speed in complex environment navigation without depth data.

Contribution

It proposes a ternary opacity model and a hybrid odometry method, significantly improving NeRF-SLAM performance using only RGB inputs.

Findings

Achieved state-of-the-art accuracy on synthetic and real datasets.

Enhanced depth rendering through the ternary opacity model.

Improved speed and robustness in SLAM tasks.

Abstract

In this work, we address the challenge of deploying Neural Radiance Field (NeRFs) in Simultaneous Localization and Mapping (SLAM) under the condition of lacking depth information, relying solely on RGB inputs. The key to unlocking the full potential of NeRF in such a challenging context lies in the integration of real-world priors. A crucial prior we explore is the binary opacity prior of 3D space with opaque objects. To effectively incorporate this prior into the NeRF framework, we introduce a ternary-type opacity (TT) model instead, which categorizes points on a ray intersecting a surface into three regions: before, on, and behind the surface. This enables a more accurate rendering of depth, subsequently improving the performance of image warping techniques. Therefore, we further propose a novel hybrid odometry (HO) scheme that merges bundle adjustment and warping-based localization. Our integrated approach of TT and HO achieves state-of-the-art performance on synthetic and real-world datasets, in terms of both speed and accuracy. This breakthrough underscores the potential of NeRF-SLAM in navigating complex environments with high fidelity.