RenderNet: Visual Relocalization Using Virtual Viewpoints in Large-Scale Indoor Environments

TL;DR

RenderNet introduces a virtual view synthesis approach that enhances visual relocalization accuracy in large-scale indoor environments by rendering features of virtual viewpoints, improving performance significantly over existing methods.

Contribution

The paper presents RenderNet, a novel virtual view synthesis method that directly renders features instead of images, improving relocalization accuracy in large-scale indoor scenes.

Findings

Achieved 7.1% improvement on Inloc dataset.

Achieved 12.2% improvement on Inloc dataset.

Effective in large-scale indoor environments.

Abstract

Visual relocalization has been a widely discussed problem in 3D vision: given a pre-constructed 3D visual map, the 6 DoF (Degrees-of-Freedom) pose of a query image is estimated. Relocalization in large-scale indoor environments enables attractive applications such as augmented reality and robot navigation. However, appearance changes fast in such environments when the camera moves, which is challenging for the relocalization system. To address this problem, we propose a virtual view synthesis-based approach, RenderNet, to enrich the database and refine poses regarding this particular scenario. Instead of rendering real images which requires high-quality 3D models, we opt to directly render the needed global and local features of virtual viewpoints and apply them in the subsequent image retrieval and feature matching operations respectively. The proposed method can largely improve the performance in large-scale indoor environments, e.g., achieving an improvement of 7.1\% and 12.2\% on the Inloc dataset.