Diffusion-Based Depth Inpainting for Transparent and Reflective Objects

TL;DR

This paper introduces DITR, a diffusion-based depth inpainting framework designed to accurately recover depth information for transparent and reflective objects, overcoming limitations of RGB-D cameras.

Contribution

The paper presents a novel two-stage diffusion-based inpainting method specifically tailored for transparent and reflective objects, with dynamic analysis of optical and geometric depth loss.

Findings

Effective depth inpainting for transparent objects

Robust adaptability demonstrated through experiments

Outperforms existing methods in accuracy

Abstract

Transparent and reflective objects, which are common in our everyday lives, present a significant challenge to 3D imaging techniques due to their unique visual and optical properties. Faced with these types of objects, RGB-D cameras fail to capture the real depth value with their accurate spatial information. To address this issue, we propose DITR, a diffusion-based Depth Inpainting framework specifically designed for Transparent and Reflective objects. This network consists of two stages, including a Region Proposal stage and a Depth Inpainting stage. DITR dynamically analyzes the optical and geometric depth loss and inpaints them automatically. Furthermore, comprehensive experimental results demonstrate that DITR is highly effective in depth inpainting tasks of transparent and reflective objects with robust adaptability.