Visible Structure Retrieval for Lightweight Image-Based Relocalisation

TL;DR

This paper introduces a neural network-based method for direct visible structure retrieval in camera relocalisation, reducing search complexity and storage needs while maintaining high accuracy in pose estimation.

Contribution

It proposes a novel neural network that directly maps images to visible scene structure, eliminating the need for image retrieval or heuristics in relocalisation.

Findings

Achieves state-of-the-art accuracy in localisation

Reduces computational and storage requirements

Enables efficient relocalisation in large scenes

Abstract

Accurate camera pose estimation from an image observation in a previously mapped environment is commonly done through structure-based methods: by finding correspondences between 2D keypoints on the image and 3D structure points in the map. In order to make this correspondence search tractable in large scenes, existing pipelines either rely on search heuristics, or perform image retrieval to reduce the search space by comparing the current image to a database of past observations. However, these approaches result in elaborate pipelines or storage requirements that grow with the number of past observations. In this work, we propose a new paradigm for making structure-based relocalisation tractable. Instead of relying on image retrieval or search heuristics, we learn a direct mapping from image observations to the visible scene structure in a compact neural network. Given a query image, a forward pass through our novel visible structure retrieval network allows obtaining the subset of 3D structure points in the map that the image views, thus reducing the search space of 2D-3D correspondences. We show that our proposed method enables performing localisation with an accuracy comparable to the state of the art, while requiring lower computational and storage footprint.