DualRefine: Self-Supervised Depth and Pose Estimation Through Iterative Epipolar Sampling and Refinement Toward Equilibrium

TL;DR

DualRefine introduces an iterative, self-supervised approach that tightly couples depth and pose estimation using a deep equilibrium model, leading to improved accuracy in depth and odometry predictions on KITTI.

Contribution

The paper presents a novel feedback loop framework that iteratively refines depth and pose estimates using epipolar geometry within a deep equilibrium model.

Findings

Achieves state-of-the-art self-supervised depth prediction on KITTI.

Surpasses existing methods in odometry estimation accuracy.

Demonstrates the effectiveness of iterative refinement in depth and pose estimation.

Abstract

Self-supervised multi-frame depth estimation achieves high accuracy by computing matching costs of pixel correspondences between adjacent frames, injecting geometric information into the network. These pixel-correspondence candidates are computed based on the relative pose estimates between the frames. Accurate pose predictions are essential for precise matching cost computation as they influence the epipolar geometry. Furthermore, improved depth estimates can, in turn, be used to align pose estimates. Inspired by traditional structure-from-motion (SfM) principles, we propose the DualRefine model, which tightly couples depth and pose estimation through a feedback loop. Our novel update pipeline uses a deep equilibrium model framework to iteratively refine depth estimates and a hidden state of feature maps by computing local matching costs based on epipolar geometry. Importantly, we used the refined depth estimates and feature maps to compute pose updates at each step. This update in the pose estimates slowly alters the epipolar geometry during the refinement process. Experimental results on the KITTI dataset demonstrate competitive depth prediction and odometry prediction performance surpassing published self-supervised baselines.