Boundary-induced and scene-aggregated network for monocular depth prediction

TL;DR

This paper introduces BS-Net, a novel network for monocular depth prediction that effectively utilizes boundary cues and scene context to improve depth accuracy, especially at edges and farthest regions, achieving state-of-the-art results.

Contribution

The paper proposes a boundary-induced and scene-aggregated network with novel modules for better depth estimation from a single RGB image.

Findings

Achieves state-of-the-art performance on NYUD v2 and iBims-1 datasets.

Effectively improves boundary accuracy and farthest region perception.

Demonstrates good generalization on the SUN-RGBD dataset.

Abstract

Monocular depth prediction is an important task in scene understanding. It aims to predict the dense depth of a single RGB image. With the development of deep learning, the performance of this task has made great improvements. However, two issues remain unresolved: (1) The deep feature encodes the wrong farthest region in a scene, which leads to a distorted 3D structure of the predicted depth; (2) The low-level features are insufficient utilized, which makes it even harder to estimate the depth near the edge with sudden depth change. To tackle these two issues, we propose the Boundary-induced and Scene-aggregated network (BS-Net). In this network, the Depth Correlation Encoder (DCE) is first designed to obtain the contextual correlations between the regions in an image, and perceive the farthest region by considering the correlations. Meanwhile, the Bottom-Up Boundary Fusion (BUBF) module is designed to extract accurate boundary that indicates depth change. Finally, the Stripe Refinement module (SRM) is designed to refine the dense depth induced by the boundary cue, which improves the boundary accuracy of the predicted depth. Several experimental results on the NYUD v2 dataset and \xff{the iBims-1 dataset} illustrate the state-of-the-art performance of the proposed approach. And the SUN-RGBD dataset is employed to evaluate the generalization of our method. Code is available at https://github.com/XuefengBUPT/BS-Net.