Loc-NeRF: Monte Carlo Localization using Neural Radiance Fields

TL;DR

Loc-NeRF introduces a real-time robot localization method that leverages pre-trained Neural Radiance Fields as maps, enabling fast, initial-pose-free localization using only an RGB camera, demonstrated on real and synthetic data.

Contribution

It is the first to combine NeRF with Monte Carlo localization for real-time, initial-pose-free robot localization in robotics.

Findings

Achieves faster localization than existing methods.

Operates in real-time using only RGB camera data.

Successfully localizes in real-world environments with NeRF maps.

Abstract

We present Loc-NeRF, a real-time vision-based robot localization approach that combines Monte Carlo localization and Neural Radiance Fields (NeRF). Our system uses a pre-trained NeRF model as the map of an environment and can localize itself in real-time using an RGB camera as the only exteroceptive sensor onboard the robot. While neural radiance fields have seen significant applications for visual rendering in computer vision and graphics, they have found limited use in robotics. Existing approaches for NeRF-based localization require both a good initial pose guess and significant computation, making them impractical for real-time robotics applications. By using Monte Carlo localization as a workhorse to estimate poses using a NeRF map model, Loc-NeRF is able to perform localization faster than the state of the art and without relying on an initial pose estimate. In addition to testing on synthetic data, we also run our system using real data collected by a Clearpath Jackal UGV and demonstrate for the first time the ability to perform real-time global localization with neural radiance fields. We make our code publicly available at https://github.com/MIT-SPARK/Loc-NeRF.