SplatLoc: 3D Gaussian Splatting-based Visual Localization for Augmented Reality

TL;DR

SplatLoc introduces a novel 3D Gaussian splatting-based method for visual localization in AR, enabling high-quality rendering and accurate pose estimation with fewer parameters and improved efficiency.

Contribution

The paper proposes a new 3D Gaussian primitive-based scene representation and a saliency-driven landmark selection algorithm for efficient and precise AR visual localization.

Findings

Achieves superior or comparable rendering quality to implicit methods.

Demonstrates high localization accuracy on standard datasets.

Reduces storage and computational requirements for AR localization.

Abstract

Visual localization plays an important role in the applications of Augmented Reality (AR), which enable AR devices to obtain their 6-DoF pose in the pre-build map in order to render virtual content in real scenes. However, most existing approaches can not perform novel view rendering and require large storage capacities for maps. To overcome these limitations, we propose an efficient visual localization method capable of high-quality rendering with fewer parameters. Specifically, our approach leverages 3D Gaussian primitives as the scene representation. To ensure precise 2D-3D correspondences for pose estimation, we develop an unbiased 3D scene-specific descriptor decoder for Gaussian primitives, distilled from a constructed feature volume. Additionally, we introduce a salient 3D landmark selection algorithm that selects a suitable primitive subset based on the saliency score for localization. We further regularize key Gaussian primitives to prevent anisotropic effects, which also improves localization performance. Extensive experiments on two widely used datasets demonstrate that our method achieves superior or comparable rendering and localization performance to state-of-the-art implicit-based visual localization approaches. Project page: \href{https://zju3dv.github.io/splatloc}{https://zju3dv.github.io/splatloc}.