OMNI-DC: Highly Robust Depth Completion with Multiresolution Depth Integration

TL;DR

OMNI-DC is a robust depth completion model that generalizes well across datasets by integrating multi-resolution depth information and employing a novel Laplacian loss, significantly reducing errors in diverse scenarios.

Contribution

The paper introduces OMNI-DC, a depth completion model with a multi-resolution depth integrator and Laplacian loss, enhancing zero-shot generalization across datasets.

Findings

Reduces depth prediction errors by up to 43% across 7 datasets.

Outperforms baseline methods in diverse and unseen data scenarios.

Demonstrates strong zero-shot generalization capabilities.

Abstract

Depth completion (DC) aims to predict a dense depth map from an RGB image and a sparse depth map. Existing DC methods generalize poorly to new datasets or unseen sparse depth patterns, limiting their real-world applications. We propose OMNI-DC, a highly robust DC model that generalizes well zero-shot to various datasets. The key design is a novel Multi-resolution Depth Integrator, allowing our model to deal with very sparse depth inputs. We also introduce a novel Laplacian loss to model the ambiguity in the training process. Moreover, we train OMNI-DC on a mixture of high-quality datasets with a scale normalization technique and synthetic depth patterns. Extensive experiments on 7 datasets show consistent improvements over baselines, reducing errors by as much as 43%. Codes and checkpoints are available at https://github.com/princeton-vl/OMNI-DC.