Deep Two-View Structure-from-Motion Revisited

TL;DR

This paper revisits deep two-view SfM by combining classical well-posed methods with deep learning, leading to improved relative pose and depth estimation over state-of-the-art approaches.

Contribution

It introduces a hybrid approach using optical flow, normalized pose estimation, and scale-invariant depth networks leveraging epipolar geometry, enhancing accuracy.

Findings

Outperforms existing methods on KITTI, MVS, Scenes11, and SUN3D datasets.

Achieves higher accuracy in relative pose estimation.

Provides more precise depth maps in two-view SfM.

Abstract

Two-view structure-from-motion (SfM) is the cornerstone of 3D reconstruction and visual SLAM. Existing deep learning-based approaches formulate the problem by either recovering absolute pose scales from two consecutive frames or predicting a depth map from a single image, both of which are ill-posed problems. In contrast, we propose to revisit the problem of deep two-view SfM by leveraging the well-posedness of the classic pipeline. Our method consists of 1) an optical flow estimation network that predicts dense correspondences between two frames; 2) a normalized pose estimation module that computes relative camera poses from the 2D optical flow correspondences, and 3) a scale-invariant depth estimation network that leverages epipolar geometry to reduce the search space, refine the dense correspondences, and estimate relative depth maps. Extensive experiments show that our method outperforms all state-of-the-art two-view SfM methods by a clear margin on KITTI depth, KITTI VO, MVS, Scenes11, and SUN3D datasets in both relative pose and depth estimation.