Depth Estimation by Combining Binocular Stereo and Monocular Structured-Light

TL;DR

This paper introduces a hybrid stereo system combining binocular stereo and monocular structured-light to improve depth estimation in indoor scenes, especially on weak texture objects, using a novel IR-based setup and deep learning guidance.

Contribution

A new hybrid stereo system integrating RGB, IR cameras, and structured-light for enhanced depth estimation in weak texture environments.

Findings

Improved depth accuracy with 28.2% error reduction compared to passive stereo.

Effective use of structured-light guidance for stereo matching.

Prototype and dataset demonstrate practical applicability.

Abstract

It is well known that the passive stereo system cannot adapt well to weak texture objects, e.g., white walls. However, these weak texture targets are very common in indoor environments. In this paper, we present a novel stereo system, which consists of two cameras (an RGB camera and an IR camera) and an IR speckle projector. The RGB camera is used both for depth estimation and texture acquisition. The IR camera and the speckle projector can form a monocular structured-light (MSL) subsystem, while the two cameras can form a binocular stereo subsystem. The depth map generated by the MSL subsystem can provide external guidance for the stereo matching networks, which can improve the matching accuracy significantly. In order to verify the effectiveness of the proposed system, we build a prototype and collect a test dataset in indoor scenes. The evaluation results show that the Bad 2.0 error of the proposed system is 28.2% of the passive stereo system when the network RAFT is used. The dataset and trained models are available at https://github.com/YuhuaXu/MonoStereoFusion.