Deep Depth Completion from Extremely Sparse Data: A Survey

TL;DR

This survey reviews deep learning methods for depth completion from sparse data, analyzing architectures, datasets, and strategies, and providing a taxonomy, performance comparison, and future research insights.

Contribution

It offers the first comprehensive review and taxonomy of deep depth completion methods, along with benchmark comparisons and discussion of future challenges.

Findings

Deep learning dominates depth completion performance.

A new taxonomy categorizes existing methods.

Benchmark results highlight current state-of-the-art models.

Abstract

Depth completion aims at predicting dense pixel-wise depth from an extremely sparse map captured from a depth sensor, e.g., LiDARs. It plays an essential role in various applications such as autonomous driving, 3D reconstruction, augmented reality, and robot navigation. Recent successes on the task have been demonstrated and dominated by deep learning based solutions. In this article, for the first time, we provide a comprehensive literature review that helps readers better grasp the research trends and clearly understand the current advances. We investigate the related studies from the design aspects of network architectures, loss functions, benchmark datasets, and learning strategies with a proposal of a novel taxonomy that categorizes existing methods. Besides, we present a quantitative comparison of model performance on three widely used benchmarks, including indoor and outdoor datasets. Finally, we discuss the challenges of prior works and provide readers with some insights for future research directions.