H-Net: Unsupervised Attention-based Stereo Depth Estimation Leveraging Epipolar Geometry

TL;DR

H-Net is an unsupervised deep learning framework that uses epipolar geometry and semantic information to improve stereo depth estimation, outperforming existing unsupervised methods and approaching supervised performance.

Contribution

Introduces H-Net, a novel unsupervised stereo depth estimation method using a Siamese autoencoder and mutual epipolar attention with semantic integration.

Findings

Outperforms state-of-the-art unsupervised methods on KITTI2015 and Cityscapes.

Effectively leverages epipolar geometry and semantic info for better depth accuracy.

Closes the gap between unsupervised and supervised stereo depth estimation.

Abstract

Depth estimation from a stereo image pair has become one of the most explored applications in computer vision, with most of the previous methods relying on fully supervised learning settings. However, due to the difficulty in acquiring accurate and scalable ground truth data, the training of fully supervised methods is challenging. As an alternative, self-supervised methods are becoming more popular to mitigate this challenge. In this paper, we introduce the H-Net, a deep-learning framework for unsupervised stereo depth estimation that leverages epipolar geometry to refine stereo matching. For the first time, a Siamese autoencoder architecture is used for depth estimation which allows mutual information between the rectified stereo images to be extracted. To enforce the epipolar constraint, the mutual epipolar attention mechanism has been designed which gives more emphasis to correspondences of features which lie on the same epipolar line while learning mutual information between the input stereo pair. Stereo correspondences are further enhanced by incorporating semantic information to the proposed attention mechanism. More specifically, the optimal transport algorithm is used to suppress attention and eliminate outliers in areas not visible in both cameras. Extensive experiments on KITTI2015 and Cityscapes show that our method outperforms the state-ofthe-art unsupervised stereo depth estimation methods while closing the gap with the fully supervised approaches.