Zero-shot Depth Completion via Test-time Alignment with Affine-invariant Depth Prior

TL;DR

This paper introduces a zero-shot depth completion method that uses a pre-trained affine-invariant depth diffusion model and test-time alignment to generalize across domains without extensive training.

Contribution

It proposes a novel approach combining a depth diffusion prior with test-time alignment for domain-generalizable depth completion.

Findings

Achieves up to 21% performance improvement over state-of-the-art methods.

Demonstrates strong generalization across various domain datasets.

Enhances spatial understanding by sharpening scene details.

Abstract

Depth completion, predicting dense depth maps from sparse depth measurements, is an ill-posed problem requiring prior knowledge. Recent methods adopt learning-based approaches to implicitly capture priors, but the priors primarily fit in-domain data and do not generalize well to out-of-domain scenarios. To address this, we propose a zero-shot depth completion method composed of an affine-invariant depth diffusion model and test-time alignment. We use pre-trained depth diffusion models as depth prior knowledge, which implicitly understand how to fill in depth for scenes. Our approach aligns the affine-invariant depth prior with metric-scale sparse measurements, enforcing them as hard constraints via an optimization loop at test-time. Our zero-shot depth completion method demonstrates generalization across various domain datasets, achieving up to a 21\% average performance improvement over the previous state-of-the-art methods while enhancing spatial understanding by sharpening scene details. We demonstrate that aligning a monocular affine-invariant depth prior with sparse metric measurements is a proven strategy to achieve domain-generalizable depth completion without relying on extensive training data. Project page: https://hyoseok1223.github.io/zero-shot-depth-completion/.