Camera Relocalization in Shadow-free Neural Radiance Fields

TL;DR

This paper introduces a two-stage pipeline using hash-encoded NeRFs and novel filtering techniques to enhance camera relocalization accuracy under varying lighting and shadow conditions.

Contribution

It presents a lighting normalization method combined with a hash-encoded NeRF and new gradient smoothing techniques for improved relocalization.

Findings

Achieves state-of-the-art relocalization accuracy under different lighting conditions.

Introduces a truncated dynamic low-pass filter (TDLF) for noise reduction in gradient computation.

Demonstrates significant improvements on multiple datasets.

Abstract

Camera relocalization is a crucial problem in computer vision and robotics. Recent advancements in neural radiance fields (NeRFs) have shown promise in synthesizing photo-realistic images. Several works have utilized NeRFs for refining camera poses, but they do not account for lighting changes that can affect scene appearance and shadow regions, causing a degraded pose optimization process. In this paper, we propose a two-staged pipeline that normalizes images with varying lighting and shadow conditions to improve camera relocalization. We implement our scene representation upon a hash-encoded NeRF which significantly boosts up the pose optimization process. To account for the noisy image gradient computing problem in grid-based NeRFs, we further propose a re-devised truncated dynamic low-pass filter (TDLF) and a numerical gradient averaging technique to smoothen the process. Experimental results on several datasets with varying lighting conditions demonstrate that our method achieves state-of-the-art results in camera relocalization under varying lighting conditions. Code and data will be made publicly available.