Depth Map Completion by Jointly Exploiting Blurry Color Images and Sparse Depth Maps

TL;DR

This paper introduces a novel method for depth map completion that jointly utilizes blurry color image sequences and sparse depth measurements, simultaneously deblurring images, estimating scene flow, and completing depth maps, achieving state-of-the-art results.

Contribution

It presents an energy minimization framework that handles blurry images and sparse depth data together, which is a significant advancement over existing methods assuming sharp images.

Findings

Achieves superior depth completion accuracy on outdoor and indoor datasets.

Effectively deblurs color images while completing depth maps.

Joint estimation improves overall scene understanding.

Abstract

We aim at predicting a complete and high-resolution depth map from incomplete, sparse and noisy depth measurements. Existing methods handle this problem either by exploiting various regularizations on the depth maps directly or resorting to learning based methods. When the corresponding color images are available, the correlation between the depth maps and the color images are used to improve the completion performance, assuming the color images are clean and sharp. However, in real world dynamic scenes, color images are often blurry due to the camera motion and the moving objects in the scene. In this paper, we propose to tackle the problem of depth map completion by jointly exploiting the blurry color image sequences and the sparse depth map measurements, and present an energy minimization based formulation to simultaneously complete the depth maps, estimate the scene flow and deblur the color images. Our experimental evaluations on both outdoor and indoor scenarios demonstrate the state-of-the-art performance of our approach.