Expanding Sparse Guidance for Stereo Matching

TL;DR

This paper introduces a sparsity expansion technique that leverages sparse LiDAR disparity cues to enhance stereo matching accuracy, improving performance across various algorithms and scenarios.

Contribution

The novel sparsity expansion method effectively utilizes sparse LiDAR data to improve stereo estimation, outperforming previous approaches in accuracy and robustness.

Findings

Outperforms previous methods by more than 2 pixel error on KITTI Stereo 2012

Achieves more than 3 pixel error improvement on KITTI Stereo 2015

Enhances existing stereo algorithms with extremely sparse cues

Abstract

The performance of image based stereo estimation suffers from lighting variations, repetitive patterns and homogeneous appearance. Moreover, to achieve good performance, stereo supervision requires sufficient densely-labeled data, which are hard to obtain. In this work, we leverage small amount of data with very sparse but accurate disparity cues from LiDAR to bridge the gap. We propose a novel sparsity expansion technique to expand the sparse cues concerning RGB images for local feature enhancement. The feature enhancement method can be easily applied to any stereo estimation algorithms with cost volume at the test stage. Extensive experiments on stereo datasets demonstrate the effectiveness and robustness across different backbones on domain adaption and self-supervision scenario. Our sparsity expansion method outperforms previous methods in terms of disparity by more than 2 pixel error on KITTI Stereo 2012 and 3 pixel error on KITTI Stereo 2015. Our approach significantly boosts the existing state-of-the-art stereo algorithms with extremely sparse cues.