Adaptive 3D Reconstruction via Diffusion Priors and Forward Curvature-Matching Likelihood Updates

TL;DR

This paper introduces a novel diffusion-based 3D reconstruction method that adaptively updates likelihoods using curvature-matching, enabling high-quality reconstructions from various input views without retraining.

Contribution

It proposes a new Forward Curvature-Matching update technique integrated with diffusion sampling, improving flexibility and efficiency in 3D point cloud reconstruction.

Findings

Achieves superior reconstruction quality on ShapeNet and CO3D datasets.

Supports single-view and multi-view inputs without retraining.

Demonstrates higher F-score and lower CD and EMD metrics.

Abstract

Reconstructing high-quality point clouds from images remains challenging in computer vision. Existing generative-model-based approaches, particularly diffusion-model approaches that directly learn the posterior, may suffer from inflexibility -- they require conditioning signals during training, support only a fixed number of input views, and need complete retraining for different measurements. Recent diffusion-based methods have attempted to address this by combining prior models with likelihood updates, but they rely on heuristic fixed step sizes for the likelihood update that lead to slow convergence and suboptimal reconstruction quality. We advance this line of approach by integrating our novel Forward Curvature-Matching (FCM) update method with diffusion sampling. Our method dynamically determines optimal step sizes using only forward automatic differentiation and finite-difference curvature estimates, enabling precise optimization of the likelihood update. This formulation enables high-fidelity reconstruction from both single-view and multi-view inputs, and supports various input modalities through simple operator substitution -- all without retraining. Experiments on ShapeNet and CO3D datasets demonstrate that our method achieves superior reconstruction quality at matched or lower NFEs, yielding higher F-score and lower CD and EMD, validating its efficiency and adaptability for practical applications. Code is available at https://github.com/Seunghyeok0715/FCM