Depth Map Prediction from a Single Image using a Multi-Scale Deep Network

TL;DR

This paper introduces a multi-scale deep learning approach for predicting depth from a single image, combining global and local information to improve accuracy and handle scale ambiguity, achieving state-of-the-art results.

Contribution

The novel multi-scale deep network architecture effectively integrates global and local cues for single-image depth prediction, addressing scale ambiguity and improving boundary accuracy.

Findings

Achieves state-of-the-art results on NYU Depth and KITTI datasets.

Effectively models depth relations using scale-invariant error.

Refines depth predictions locally for detailed boundaries.

Abstract

Predicting depth is an essential component in understanding the 3D geometry of a scene. While for stereo images local correspondence suffices for estimation, finding depth relations from a single image is less straightforward, requiring integration of both global and local information from various cues. Moreover, the task is inherently ambiguous, with a large source of uncertainty coming from the overall scale. In this paper, we present a new method that addresses this task by employing two deep network stacks: one that makes a coarse global prediction based on the entire image, and another that refines this prediction locally. We also apply a scale-invariant error to help measure depth relations rather than scale. By leveraging the raw datasets as large sources of training data, our method achieves state-of-the-art results on both NYU Depth and KITTI, and matches detailed depth boundaries without the need for superpixelation.