Relative Depth Order Estimation Using Multi-scale Densely Connected Convolutional Networks

TL;DR

This paper investigates how different contextual cues contribute to relative depth order estimation in monocular images and proposes a multi-scale densely-connected network focusing on local structures, achieving competitive results with limited training data.

Contribution

It introduces a multi-scale densely-connected network that emphasizes local context for depth order estimation, revealing local cues are most influential.

Findings

Local context contributes most to depth estimation accuracy.

Global scene context has limited impact.

Proposed method outperforms or matches state-of-the-art with less training data.

Abstract

We study the problem of estimating the relative depth order of point pairs in a monocular image. Recent advances mainly focus on using deep convolutional neural networks (DCNNs) to learn and infer the ordinal information from multiple contextual information of the points pair such as global scene context, local contextual information, and the locations. However, it remains unclear how much each context contributes to the task. To address this, we first examine the contribution of each context cue [1], [2] to the performance in the context of depth order estimation. We find out the local context surrounding the points pair contributes the most and the global scene context helps little. Based on the findings, we propose a simple method, using a multi-scale densely-connected network to tackle the task. Instead of learning the global structure, we dedicate to explore the local structure by learning to regress from regions of multiple sizes around the point pairs. Moreover, we use the recent densely connected network [3] to encourage substantial feature reuse as well as deepen our network to boost the performance. We show in experiments that the results of our approach is on par with or better than the state-of-the-art methods with the benefit of using only a small number of training data.