NeRF-Loc: Visual Localization with Conditional Neural Radiance Field
Jianlin Liu, Qiang Nie, Yong Liu, Chengjie Wang

TL;DR
NeRF-Loc introduces a novel visual re-localization approach using a conditional neural radiance field for continuous 3D descriptors and neural rendering, enhancing accuracy and robustness over existing methods.
Contribution
The paper presents a unified framework combining feature matching and scene coordinate regression with a style adaptation layer for improved localization.
Findings
Achieves higher localization accuracy than existing learning-based methods.
Supports continuous 3D descriptor generation and neural rendering.
Improves robustness with appearance adaptation layer.
Abstract
We propose a novel visual re-localization method based on direct matching between the implicit 3D descriptors and the 2D image with transformer. A conditional neural radiance field(NeRF) is chosen as the 3D scene representation in our pipeline, which supports continuous 3D descriptors generation and neural rendering. By unifying the feature matching and the scene coordinate regression to the same framework, our model learns both generalizable knowledge and scene prior respectively during two training stages. Furthermore, to improve the localization robustness when domain gap exists between training and testing phases, we propose an appearance adaptation layer to explicitly align styles between the 3D model and the query image. Experiments show that our method achieves higher localization accuracy than other learning-based approaches on multiple benchmarks. Code is available at…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAdvanced Vision and Imaging · Robotics and Sensor-Based Localization · Advanced Image and Video Retrieval Techniques
MethodsALIGN
