Robust Full-FoV Depth Estimation in Tele-wide Camera System

TL;DR

This paper introduces a hierarchical hourglass network that combines traditional stereo-matching and deep learning for robust, full-field-of-view depth estimation in tele-wide camera systems, improving accuracy and robustness.

Contribution

It proposes a novel hierarchical hourglass network integrating stereo-matching and DNNs for enhanced robustness in full-FoV depth estimation.

Findings

Outperforms state-of-the-art methods on standard datasets

Produces more robust depth estimates on wild test images

Achieves better subjective and quantitative depth quality

Abstract

Tele-wide camera system with different Field of View (FoV) lenses becomes very popular in recent mobile devices. Usually it is difficult to obtain full-FoV depth based on traditional stereo-matching methods. Pure Deep Neural Network (DNN) based depth estimation methods can obtain full-FoV depth, but have low robustness for scenarios which are not covered by training dataset. In this paper, to address the above problems we propose a hierarchical hourglass network for robust full-FoV depth estimation in tele-wide camera system, which combines the robustness of traditional stereo-matching methods with the accuracy of DNN. More specifically, the proposed network comprises three major modules: single image depth prediction module infers initial depth from input color image, depth propagation module propagates traditional stereo-matching tele-FoV depth to surrounding regions, and depth combination module fuses the initial depth with the propagated depth to generate final output. Each of these modules employs an hourglass model, which is a kind of encoder-decoder structure with skip connections. Experimental results compared with state-of-the-art depth estimation methods demonstrate that our method not only produces robust and better subjective depth quality on wild test images, but also obtains better quantitative results on standard datasets.