Deep Stereo Matching with Explicit Cost Aggregation Sub-Architecture

TL;DR

This paper introduces a novel learning-based cost aggregation sub-architecture for stereo matching, improving accuracy by explicitly modeling proposal generation and selection within an end-to-end neural network.

Contribution

It proposes a two-stream network for cost aggregation that utilizes structure-aware proposals, enhancing stereo matching performance over existing methods.

Findings

Outperforms state-of-the-art on KITTI and Scene Flow datasets

Effective integration of structure information improves matching accuracy

End-to-end trainable architecture enhances stereo matching results

Abstract

Deep neural networks have shown excellent performance for stereo matching. Many efforts focus on the feature extraction and similarity measurement of the matching cost computation step while less attention is paid on cost aggregation which is crucial for stereo matching. In this paper, we present a learning-based cost aggregation method for stereo matching by a novel sub-architecture in the end-to-end trainable pipeline. We reformulate the cost aggregation as a learning process of the generation and selection of cost aggregation proposals which indicate the possible cost aggregation results. The cost aggregation sub-architecture is realized by a two-stream network: one for the generation of cost aggregation proposals, the other for the selection of the proposals. The criterion for the selection is determined by the low-level structure information obtained from a light convolutional network. The two-stream network offers a global view guidance for the cost aggregation to rectify the mismatching value stemming from the limited view of the matching cost computation. The comprehensive experiments on challenge datasets such as KITTI and Scene Flow show that our method outperforms the state-of-the-art methods.