Computing the Stereo Matching Cost with a Convolutional Neural Network

TL;DR

This paper introduces a CNN-based approach for stereo matching that predicts patch similarity, leading to highly accurate depth estimation, and achieves top performance on the KITTI dataset.

Contribution

It presents a novel CNN-based stereo matching cost computation method combined with advanced refinement techniques, setting a new benchmark in accuracy.

Findings

Error rate of 2.61% on KITTI dataset

Outperforms previous methods in stereo matching accuracy

Effective integration of CNN with cost aggregation and consistency checks

Abstract

We present a method for extracting depth information from a rectified image pair. We train a convolutional neural network to predict how well two image patches match and use it to compute the stereo matching cost. The cost is refined by cross-based cost aggregation and semiglobal matching, followed by a left-right consistency check to eliminate errors in the occluded regions. Our stereo method achieves an error rate of 2.61 % on the KITTI stereo dataset and is currently (August 2014) the top performing method on this dataset.