CUPID: Generative 3D Reconstruction via Joint Object and Pose Modeling

TL;DR

Cupid is a novel generative framework for 3D reconstruction that jointly models object shape and camera pose, achieving high accuracy and extending naturally to multi-view and scene-level tasks.

Contribution

Introduces Cupid, a two-stage flow-based generative model that jointly estimates object shape and camera pose for improved 3D reconstruction.

Findings

Outperforms state-of-the-art methods by over 3 dB PSNR

Achieves 10% improvement in Chamfer Distance

Extends to multi-view and scene-level reconstruction without additional optimization

Abstract

We introduce Cupid, a generative 3D reconstruction framework that jointly models the full distribution over both canonical objects and camera poses. Our two-stage flow-based model first generates a coarse 3D structure and 2D-3D correspondences to estimate the camera pose robustly. Conditioned on this pose, a refinement stage injects pixel-aligned image features directly into the generative process, marrying the rich prior of a generative model with the geometric fidelity of reconstruction. This strategy achieves exceptional faithfulness, outperforming state-of-the-art reconstruction methods by over 3 dB PSNR and 10% in Chamfer Distance. As a unified generative model that decouples the object and camera pose, Cupid naturally extends to multi-view and scene-level reconstruction tasks without requiring post-hoc optimization or fine-tuning.