Towards Open World NeRF-Based SLAM

TL;DR

This paper enhances NeRF-based SLAM by improving robustness, accuracy, and environment generalization, enabling more open-world applications with better performance and mapping capabilities.

Contribution

It introduces improvements to NICE-SLAM, including better uncertainty handling, motion integration, and explicit foreground-background modeling, to achieve more accurate and flexible SLAM in unconstrained environments.

Findings

Tracking accuracy improved by 85% to 97%.

Enhanced mapping in environments extending beyond predefined grids.

Achieved more open-world-capable NeRF-based SLAM.

Abstract

Neural Radiance Fields (NeRFs) offer versatility and robustness in map representations for Simultaneous Localization and Mapping (SLAM) tasks. This paper extends NICE-SLAM, a recent state-of-the-art NeRF-based SLAM algorithm capable of producing high quality NeRF maps. However, depending on the hardware used, the required number of iterations to produce these maps often makes NICE-SLAM run at less than real time. Additionally, the estimated trajectories fail to be competitive with classical SLAM approaches. Finally, NICE-SLAM requires a grid covering the considered environment to be defined prior to runtime, making it difficult to extend into previously unseen scenes. This paper seeks to make NICE-SLAM more open-world-capable by improving the robustness and tracking accuracy, and generalizing the map representation to handle unconstrained environments. This is done by improving measurement uncertainty handling, incorporating motion information, and modelling the map as having an explicit foreground and background. It is shown that these changes are able to improve tracking accuracy by 85% to 97% depending on the available resources, while also improving mapping in environments with visual information extending outside of the predefined grid.