RePoseD: Efficient Relative Pose Estimation With Known Depth Information

TL;DR

This paper introduces RePoseD, a novel framework that leverages monocular depth estimates for more accurate and efficient relative pose estimation between two cameras, outperforming existing methods.

Contribution

The paper presents new depth-aware solvers that jointly estimate scale and pose, improving speed and accuracy over prior approaches across various camera configurations.

Findings

Outperforms state-of-the-art depth-aware solvers in speed and accuracy.

Effective across multiple datasets and monocular depth estimation methods.

Provides guidelines on which solver to use in different scenarios.

Abstract

Recent advances in monocular depth estimation methods (MDE) and their improved accuracy open new possibilities for their applications. In this paper, we investigate how monocular depth estimates can be used for relative pose estimation. In particular, we are interested in answering the question whether using MDEs improves results over traditional point-based methods. We propose a novel framework for estimating the relative pose of two cameras from point correspondences with associated monocular depths. Since depth predictions are typically defined up to an unknown scale or even both unknown scale and shift parameters, our solvers jointly estimate the scale or both the scale and shift parameters along with the relative pose. We derive efficient solvers considering different types of depths for three camera configurations: (1) two calibrated cameras, (2) two cameras with an unknown shared focal length, and (3) two cameras with unknown different focal lengths. Our new solvers outperform state-of-the-art depth-aware solvers in terms of speed and accuracy. In extensive real experiments on multiple datasets and with various MDEs, we discuss which depth-aware solvers are preferable in which situation. The code will be made publicly available.