Decoupling Features and Coordinates for Few-shot RGB Relocalization

TL;DR

This paper introduces a decoupled approach for camera relocalization that separates feature extraction, coordinate regression, and pose estimation, enabling effective few-shot adaptation to new scenes with minimal data.

Contribution

It proposes a novel decoupled framework that learns view-insensitive features independently, improving few-shot scene adaptation in camera relocalization tasks.

Findings

Outperforms state-of-the-art methods on multiple scenes.

Achieves higher accuracy with fewer training samples.

Demonstrates robustness across diverse visual conditions.

Abstract

Cross-scene model adaption is crucial for camera relocalization in real scenarios. It is often preferable that a pre-learned model can be fast adapted to a novel scene with as few training samples as possible. The existing state-of-the-art approaches, however, can hardly support such few-shot scene adaption due to the entangling of image feature extraction and scene coordinate regression. To address this issue, we approach camera relocalization with a decoupled solution where feature extraction, coordinate regression, and pose estimation are performed separately. Our key insight is that feature encoder used for coordinate regression should be learned by removing the distracting factor of coordinate systems, such that feature encoder is learned from multiple scenes for general feature representation and more important, view-insensitive capability. With this feature prior, and combined with a coordinate regressor, few-shot observations in a new scene are much easier to connect with the 3D world than the one with existing integrated solution. Experiments have shown the superiority of our approach compared to the state-of-the-art methods, producing higher accuracy on several scenes with diverse visual appearance and viewpoint distribution.