R-SCoRe: Revisiting Scene Coordinate Regression for Robust Large-Scale Visual Localization

TL;DR

This paper enhances scene coordinate regression for visual localization by introducing novel encoding, data augmentation, and architecture improvements, achieving state-of-the-art accuracy on large-scale, challenging datasets with smaller map sizes.

Contribution

It presents a covisibility graph-based encoding, depth-adjusted loss, and architecture revisits that significantly improve SCR robustness and accuracy without large maps or 3D supervision.

Findings

Achieves 10× higher accuracy on Aachen Day-Night compared to previous SCR methods.

Requires at least 5× smaller map sizes while maintaining superior accuracy.

State-of-the-art performance on large-scale datasets without ensemble or 3D supervision.

Abstract

Learning-based visual localization methods that use scene coordinate regression (SCR) offer the advantage of smaller map sizes. However, on datasets with complex illumination changes or image-level ambiguities, it remains a less robust alternative to feature matching methods. This work aims to close the gap. We introduce a covisibility graph-based global encoding learning and data augmentation strategy, along with a depth-adjusted reprojection loss to facilitate implicit triangulation. Additionally, we revisit the network architecture and local feature extraction module. Our method achieves state-of-the-art on challenging large-scale datasets without relying on network ensembles or 3D supervision. On Aachen Day-Night, we are 10$\times$ more accurate than previous SCR methods with similar map sizes and require at least 5$\times$ smaller map sizes than any other SCR method while still delivering superior accuracy. Code is available at: https://github.com/cvg/scrstudio .