Reprojection Errors as Prompts for Efficient Scene Coordinate Regression

TL;DR

This paper introduces an error-guided feature selection method using reprojection errors and SAM to improve scene coordinate regression by focusing on reliable regions, leading to better localization accuracy.

Contribution

The paper proposes a novel error-guided feature selection mechanism that filters out problematic areas in SCR training, enhancing performance over existing methods.

Findings

Outperforms existing SCR methods on Cambridge Landmarks and Indoor6 datasets.

Effectively filters out dynamic and texture-less regions during training.

Improves localization accuracy by focusing on low reprojection error areas.

Abstract

Scene coordinate regression (SCR) methods have emerged as a promising area of research due to their potential for accurate visual localization. However, many existing SCR approaches train on samples from all image regions, including dynamic objects and texture-less areas. Utilizing these areas for optimization during training can potentially hamper the overall performance and efficiency of the model. In this study, we first perform an in-depth analysis to validate the adverse impacts of these areas. Drawing inspiration from our analysis, we then introduce an error-guided feature selection (EGFS) mechanism, in tandem with the use of the Segment Anything Model (SAM). This mechanism seeds low reprojection areas as prompts and expands them into error-guided masks, and then utilizes these masks to sample points and filter out problematic areas in an iterative manner. The experiments demonstrate that our method outperforms existing SCR approaches that do not rely on 3D information on the Cambridge Landmarks and Indoor6 datasets.