Depth from Camera Motion and Object Detection

TL;DR

This paper introduces a neural network model, DBox, for estimating object depth from camera motion and detection data, validated on a new extensive dataset and multiple benchmarks, advancing monocular depth estimation in robotics and mobile applications.

Contribution

The paper presents a novel recurrent neural network, DBox, for depth estimation from uncalibrated camera motion and detection, along with a new extensible dataset, ODMD, for training and benchmarking.

Findings

DBox achieves state-of-the-art results on robotics and driving benchmarks.

ODMD dataset contains 21,600 examples for robust training and evaluation.

DBox successfully estimates object depth using a smartphone camera.

Abstract

This paper addresses the problem of learning to estimate the depth of detected objects given some measurement of camera motion (e.g., from robot kinematics or vehicle odometry). We achieve this by 1) designing a recurrent neural network (DBox) that estimates the depth of objects using a generalized representation of bounding boxes and uncalibrated camera movement and 2) introducing the Object Depth via Motion and Detection Dataset (ODMD). ODMD training data are extensible and configurable, and the ODMD benchmark includes 21,600 examples across four validation and test sets. These sets include mobile robot experiments using an end-effector camera to locate objects from the YCB dataset and examples with perturbations added to camera motion or bounding box data. In addition to the ODMD benchmark, we evaluate DBox in other monocular application domains, achieving state-of-the-art results on existing driving and robotics benchmarks and estimating the depth of objects using a camera phone.