SGLoc: Semantic Localization System for Camera Pose Estimation from 3D Gaussian Splatting Representation

TL;DR

SGLoc is a new camera localization system that uses semantic information and 3D Gaussian Splatting to estimate and refine camera poses without needing initial pose guesses, showing superior results on standard datasets.

Contribution

It introduces a novel semantic-based localization method leveraging 3D Gaussian Splatting and multi-level pose regression without prior pose information.

Findings

Outperforms baselines on 12scenes and 7scenes datasets.

Effectively estimates camera pose without initial priors.

Utilizes semantic matching for robust localization.

Abstract

We propose SGLoc, a novel localization system that directly regresses camera poses from 3D Gaussian Splatting (3DGS) representation by leveraging semantic information. Our method utilizes the semantic relationship between 2D image and 3D scene representation to estimate the 6DoF pose without prior pose information. In this system, we introduce a multi-level pose regression strategy that progressively estimates and refines the pose of query image from the global 3DGS map, without requiring initial pose priors. Moreover, we introduce a semantic-based global retrieval algorithm that establishes correspondences between 2D (image) and 3D (3DGS map). By matching the extracted scene semantic descriptors of 2D query image and 3DGS semantic representation, we align the image with the local region of the global 3DGS map, thereby obtaining a coarse pose estimation. Subsequently, we refine the coarse pose by iteratively optimizing the difference between the query image and the rendered image from 3DGS. Our SGLoc demonstrates superior performance over baselines on 12scenes and 7scenes datasets, showing excellent capabilities in global localization without initial pose prior. Code will be available at https://github.com/IRMVLab/SGLoc.