EdgeStereo: A Context Integrated Residual Pyramid Network for Stereo Matching

TL;DR

EdgeStereo is a multi-task neural network that jointly predicts disparity and edge maps, using a context pyramid and residual refinement to improve stereo matching accuracy, especially in challenging regions.

Contribution

The paper introduces EdgeStereo, a novel multi-task network that integrates edge detection with disparity estimation using a context pyramid and residual refinement for enhanced stereo matching.

Findings

Achieves state-of-the-art results on KITTI Stereo and Scene Flow benchmarks.

Effectively preserves subtle details and boundaries in disparity maps.

Demonstrates mutual benefits of stereo matching and edge detection in a unified model.

Abstract

Recent convolutional neural networks, especially end-to-end disparity estimation models, achieve remarkable performance on stereo matching task. However, existed methods, even with the complicated cascade structure, may fail in the regions of non-textures, boundaries and tiny details. Focus on these problems, we propose a multi-task network EdgeStereo that is composed of a backbone disparity network and an edge sub-network. Given a binocular image pair, our model enables end-to-end prediction of both disparity map and edge map. Basically, we design a context pyramid to encode multi-scale context information in disparity branch, followed by a compact residual pyramid for cascaded refinement. To further preserve subtle details, our EdgeStereo model integrates edge cues by feature embedding and edge-aware smoothness loss regularization. Comparative results demonstrates that stereo matching and edge detection can help each other in the unified model. Furthermore, our method achieves state-of-art performance on both KITTI Stereo and Scene Flow benchmarks, which proves the effectiveness of our design.