RealDiff: Real-world 3D Shape Completion using Self-Supervised Diffusion Models

TL;DR

RealDiff introduces a self-supervised diffusion-based framework for 3D point cloud completion that effectively handles real-world noisy data without synthetic training, outperforming existing methods.

Contribution

The paper presents a novel self-supervised diffusion model for real-world 3D shape completion, leveraging geometric cues and silhouette/depth regularization to improve generalization.

Findings

Outperforms state-of-the-art methods on real-world datasets

Effectively handles noisy and incomplete point clouds

Uses geometric cues and silhouette/depth regularization

Abstract

Point cloud completion aims to recover the complete 3D shape of an object from partial observations. While approaches relying on synthetic shape priors achieved promising results in this domain, their applicability and generalizability to real-world data are still limited. To tackle this problem, we propose a self-supervised framework, namely RealDiff, that formulates point cloud completion as a conditional generation problem directly on real-world measurements. To better deal with noisy observations without resorting to training on synthetic data, we leverage additional geometric cues. Specifically, RealDiff simulates a diffusion process at the missing object parts while conditioning the generation on the partial input to address the multimodal nature of the task. We further regularize the training by matching object silhouettes and depth maps, predicted by our method, with the externally estimated ones. Experimental results show that our method consistently outperforms state-of-the-art methods in real-world point cloud completion.