Rendering Stable Features Improves Sampling-Based Localisation with Neural Radiance Fields

TL;DR

This paper demonstrates that rendering stable features in NeRF-based sampling localization significantly improves accuracy and efficiency, reducing the number of network forward passes needed by ten times.

Contribution

It introduces the use of stable features in NeRF sampling-based localization, showing substantial improvements over traditional feature matching methods.

Findings

Stable features lead to more accurate localization.

Rendering stable features reduces computational cost tenfold.

Sampling-based NeRF localization benefits from stable feature integration.

Abstract

Neural radiance fields (NeRFs) are a powerful tool for implicit scene representations, allowing for differentiable rendering and the ability to make predictions about unseen viewpoints. There has been growing interest in object and scene-based localisation using NeRFs, with a number of recent works relying on sampling-based or Monte-Carlo localisation schemes. Unfortunately, these can be extremely computationally expensive, requiring multiple network forward passes to infer camera or object pose. To alleviate this, a variety of sampling strategies have been applied, many relying on keypoint recognition techniques from classical computer vision. This work conducts a systematic empirical comparison of these approaches and shows that in contrast to conventional feature matching approaches for geometry-based localisation, sampling-based localisation using NeRFs benefits significantly from stable features. Results show that rendering stable features provides significantly better estimation with a tenfold reduction in the number of forward passes required.