Depth Completion as Parameter-Efficient Test-Time Adaptation

TL;DR

CAPA is a parameter-efficient test-time adaptation method for depth completion that fine-tunes pre-trained 3D models using sparse cues, improving accuracy and robustness without overfitting.

Contribution

It introduces a novel, model-agnostic framework that updates minimal parameters at inference time, leveraging sparse geometric cues and sequence-level adaptation for depth completion.

Findings

Achieves state-of-the-art results on indoor and outdoor datasets.

Effectively grounds geometric priors with minimal parameter updates.

Enhances robustness and multi-frame consistency in videos.

Abstract

We introduce CAPA, a parameter-efficient test-time optimization framework that adapts pre-trained 3D foundation models (FMs) for depth completion, using sparse geometric cues. Unlike prior methods that train task-specific encoders for auxiliary inputs, which often overfit and generalize poorly, CAPA freezes the FM backbone. Instead, it updates only a minimal set of parameters using Parameter-Efficient Fine-Tuning (e.g. LoRA or VPT), guided by gradients calculated directly from the sparse observations available at inference time. This approach effectively grounds the foundation model's geometric prior in the scene-specific measurements, correcting distortions and misplaced structures. For videos, CAPA introduces sequence-level parameter sharing, jointly adapting all frames to exploit temporal correlations, improve robustness, and enforce multi-frame consistency. CAPA is model-agnostic, compatible with any ViT-based FM, and achieves state-of-the-art results across diverse condition patterns on both indoor and outdoor datasets. Project page: research.nvidia.com/labs/dvl/projects/capa.