Gaussian Splatting Feature Fields for Privacy-Preserving Visual Localization

TL;DR

This paper introduces Gaussian Splatting Feature Fields (GSFFs), a novel scene representation combining explicit geometry and implicit features for accurate, privacy-preserving visual localization, achieving state-of-the-art results.

Contribution

The paper proposes GSFFs, a new scene representation that integrates 3D Gaussian Splatting with feature fields and segmentation for privacy-preserving localization.

Findings

Achieves state-of-the-art localization accuracy on multiple datasets.

Effectively combines explicit geometry with implicit features for privacy.

Enables privacy-preserving segmentation-based localization.

Abstract

Visual localization is the task of estimating a camera pose in a known environment. In this paper, we utilize 3D Gaussian Splatting (3DGS)-based representations for accurate and privacy-preserving visual localization. We propose Gaussian Splatting Feature Fields (GSFFs), a scene representation for visual localization that combines an explicit geometry model (3DGS) with an implicit feature field. We leverage the dense geometric information and differentiable rasterization algorithm from 3DGS to learn robust feature representations grounded in 3D. In particular, we align a 3D scale-aware feature field and a 2D feature encoder in a common embedding space through a contrastive framework. Using a 3D structure-informed clustering procedure, we further regularize the representation learning and seamlessly convert the features to segmentations, which can be used for privacy-preserving visual localization. Pose refinement, which involves aligning either feature maps or segmentations from a query image with those rendered from the GSFFs scene representation, is used to achieve localization. The resulting privacy- and non-privacy-preserving localization pipelines, evaluated on multiple real-world datasets, show state-of-the-art performances.