MeshLoc: Mesh-Based Visual Localization

TL;DR

MeshLoc introduces a flexible, mesh-based visual localization method that achieves state-of-the-art accuracy without relying on feature matching between images, using dense 3D models and rendering techniques.

Contribution

The paper presents a novel mesh-based scene representation for visual localization that eliminates the need for feature matching, enabling flexible and efficient pose estimation.

Findings

State-of-the-art localization accuracy with mesh-based representations

Competitive results using features on mesh renderings without neural rendering

Effective localization even with raw scene geometry without color or texture

Abstract

Visual localization, i.e., the problem of camera pose estimation, is a central component of applications such as autonomous robots and augmented reality systems. A dominant approach in the literature, shown to scale to large scenes and to handle complex illumination and seasonal changes, is based on local features extracted from images. The scene representation is a sparse Structure-from-Motion point cloud that is tied to a specific local feature. Switching to another feature type requires an expensive feature matching step between the database images used to construct the point cloud. In this work, we thus explore a more flexible alternative based on dense 3D meshes that does not require features matching between database images to build the scene representation. We show that this approach can achieve state-of-the-art results. We further show that surprisingly competitive results can be obtained when extracting features on renderings of these meshes, without any neural rendering stage, and even when rendering raw scene geometry without color or texture. Our results show that dense 3D model-based representations are a promising alternative to existing representations and point to interesting and challenging directions for future research.