MonSter: Awakening the Mono in Stereo

TL;DR

This paper introduces a novel two-camera system that combines stereo and monocular depth estimation methods, improving accuracy and enabling online self-calibration in real-world scenes.

Contribution

The work proposes a combined stereo-monocular system with a self-calibration strategy, enhancing depth estimation accuracy and robustness over traditional single-method approaches.

Findings

Improved depth accuracy through combined stereo and monocular methods

Effective online self-calibration achieved via consistency enforcement

Prototype demonstrates practical benefits in real-world scenes

Abstract

Passive depth estimation is among the most long-studied fields in computer vision. The most common methods for passive depth estimation are either a stereo or a monocular system. Using the former requires an accurate calibration process, and has a limited effective range. The latter, which does not require extrinsic calibration but generally achieves inferior depth accuracy, can be tuned to achieve better results in part of the depth range. In this work, we suggest combining the two frameworks. We propose a two-camera system, in which the cameras are used jointly to extract a stereo depth and individually to provide a monocular depth from each camera. The combination of these depth maps leads to more accurate depth estimation. Moreover, enforcing consistency between the extracted maps leads to a novel online self-calibration strategy. We present a prototype camera that demonstrates the benefits of the proposed combination, for both self-calibration and depth reconstruction in real-world scenes.