Amodal Depth Anything: Amodal Depth Estimation in the Wild

TL;DR

This paper introduces a new approach for amodal depth estimation in natural scenes, utilizing a large-scale dataset and novel models to improve generalization and accuracy in predicting occluded object depths.

Contribution

The paper presents a new formulation focusing on relative depth prediction, a large-scale dataset ADIW, and two innovative models, Amodal-DAV2 and Amodal-DepthFM, for improved amodal depth estimation.

Findings

Achieved 69.5% improvement over previous state-of-the-art

Developed a scalable pipeline for dataset creation using pre-trained models

Demonstrated the effectiveness of models in generating plausible occluded depth structures

Abstract

Amodal depth estimation aims to predict the depth of occluded (invisible) parts of objects in a scene. This task addresses the question of whether models can effectively perceive the geometry of occluded regions based on visible cues. Prior methods primarily rely on synthetic datasets and focus on metric depth estimation, limiting their generalization to real-world settings due to domain shifts and scalability challenges. In this paper, we propose a novel formulation of amodal depth estimation in the wild, focusing on relative depth prediction to improve model generalization across diverse natural images. We introduce a new large-scale dataset, Amodal Depth In the Wild (ADIW), created using a scalable pipeline that leverages segmentation datasets and compositing techniques. Depth maps are generated using large pre-trained depth models, and a scale-and-shift alignment strategy is employed to refine and blend depth predictions, ensuring consistency in ground-truth annotations. To tackle the amodal depth task, we present two complementary frameworks: Amodal-DAV2, a deterministic model based on Depth Anything V2, and Amodal-DepthFM, a generative model that integrates conditional flow matching principles. Our proposed frameworks effectively leverage the capabilities of large pre-trained models with minimal modifications to achieve high-quality amodal depth predictions. Experiments validate our design choices, demonstrating the flexibility of our models in generating diverse, plausible depth structures for occluded regions. Our method achieves a 69.5% improvement in accuracy over the previous SoTA on the ADIW dataset.