Deep Robust Single Image Depth Estimation Neural Network Using Scene Understanding

TL;DR

This paper introduces a two-stage deep learning framework for single image depth estimation that combines scene understanding with specialized depth networks, achieving high accuracy and efficiency across indoor and outdoor scenes.

Contribution

The paper presents a novel two-stage framework with scene classification and multi-task depth networks, improving robustness and accuracy in diverse environments.

Findings

Achieves competitive accuracy on NYU dataset.

Demonstrates good performance on ScanNet and KITTI datasets.

Outperforms ROB 2018 submissions in robustness.

Abstract

Single image depth estimation (SIDE) plays a crucial role in 3D computer vision. In this paper, we propose a two-stage robust SIDE framework that can perform blind SIDE for both indoor and outdoor scenes. At the first stage, the scene understanding module will categorize the RGB image into different depth-ranges. We introduce two different scene understanding modules based on scene classification and coarse depth estimation respectively. At the second stage, SIDE networks trained by the images of specific depth-range are applied to obtain an accurate depth map. In order to improve the accuracy, we further design a multi-task encoding-decoding SIDE network DS-SIDENet based on depthwise separable convolutions. DS-SIDENet is optimized to minimize both depth classification and depth regression losses. This improves the accuracy compared to a single-task SIDE network. Experimental results demonstrate that training DS-SIDENet on an individual dataset such as NYU achieves competitive performance to the state-of-art methods with much better efficiency. Ours proposed robust SIDE framework also shows good performance for the ScanNet indoor images and KITTI outdoor images simultaneously. It achieves the top performance compared to the Robust Vision Challenge (ROB) 2018 submissions.