Cascade Residual Learning: A Two-stage Convolutional Neural Network for Stereo Matching

TL;DR

This paper introduces a two-stage cascade CNN architecture for stereo matching, improving disparity quality especially in challenging regions, and achieves state-of-the-art results on the KITTI 2015 benchmark.

Contribution

The paper proposes a novel cascade residual learning framework that refines disparity maps through two interconnected CNN stages, enhancing detail and accuracy over previous methods.

Findings

Achieves state-of-the-art performance on KITTI 2015 stereo benchmark.

Outperforms prior methods significantly in disparity accuracy.

Demonstrates the effectiveness of residual learning in stereo matching.

Abstract

Leveraging on the recent developments in convolutional neural networks (CNNs), matching dense correspondence from a stereo pair has been cast as a learning problem, with performance exceeding traditional approaches. However, it remains challenging to generate high-quality disparities for the inherently ill-posed regions. To tackle this problem, we propose a novel cascade CNN architecture composing of two stages. The first stage advances the recently proposed DispNet by equipping it with extra up-convolution modules, leading to disparity images with more details. The second stage explicitly rectifies the disparity initialized by the first stage; it couples with the first-stage and generates residual signals across multiple scales. The summation of the outputs from the two stages gives the final disparity. As opposed to directly learning the disparity at the second stage, we show that residual learning provides more effective refinement. Moreover, it also benefits the training of the overall cascade network. Experimentation shows that our cascade residual learning scheme provides state-of-the-art performance for matching stereo correspondence. By the time of the submission of this paper, our method ranks first in the KITTI 2015 stereo benchmark, surpassing the prior works by a noteworthy margin.