An Adaptive Framework for Learning Unsupervised Depth Completion

TL;DR

This paper introduces an adaptive framework that improves unsupervised depth completion by dynamically estimating co-visibility and regularization, leading to better performance without extra parameters or inference time.

Contribution

It proposes a novel annealing-based adaptive weighting scheme that unifies co-visibility and regularization estimation in unsupervised depth completion.

Findings

Enhanced depth completion accuracy on benchmark datasets.

Improved existing methods without additional trainable parameters.

Maintained inference efficiency while boosting performance.

Abstract

We present a method to infer a dense depth map from a color image and associated sparse depth measurements. Our main contribution lies in the design of an annealing process for determining co-visibility (occlusions, disocclusions) and the degree of regularization to impose on the model. We show that regularization and co-visibility are related via the fitness (residual) of model to data and both can be unified into a single framework to improve the learning process. Our method is an adaptive weighting scheme that guides optimization by measuring the residual at each pixel location over each training step for (i) estimating a soft visibility mask and (ii) determining the amount of regularization. We demonstrate the effectiveness our method by applying it to several recent unsupervised depth completion methods and improving their performance on public benchmark datasets, without incurring additional trainable parameters or increase in inference time. Code available at: https://github.com/alexklwong/adaframe-depth-completion.