SCORP: Scene-Consistent Object Refinement via Proxy Generation and Tuning

TL;DR

SCORP is a novel 3D object refinement framework that uses generative priors and proxy tuning to recover detailed object geometry and appearance in scene reconstructions, especially when views are missing.

Contribution

Introduces SCORP, a two-stage proxy generation and tuning method that enhances 3D object reconstruction fidelity using generative priors and spatial alignment.

Findings

Outperforms recent state-of-the-art methods on view synthesis tasks

Achieves significant improvements in geometry completion accuracy

Maintains scene and object consistency across unseen views

Abstract

Viewpoint missing of objects is common in scene reconstruction, as camera paths typically prioritize capturing the overall scene structure rather than individual objects. This makes it highly challenging to achieve high-fidelity object-level modeling while maintaining accurate scene-level representation. Addressing this issue is critical for advancing downstream tasks requiring high-fidelity object reconstruction. In this paper, we introduce Scene-Consistent Object Refinement via Proxy Generation and Tuning (SCORP), a novel 3D enhancement framework that leverages 3D generative priors to recover fine-grained object geometry and appearance under missing views. Starting with proxy generation by substituting degraded objects using a 3D generation model, SCORP then progressively refines geometry and texture by aligning each proxy to its degraded counterpart in 7-DoF pose, followed by correcting spatial and appearance inconsistencies through registration-constrained enhancement. This two-stage proxy tuning ensures the high-fidelity geometry and appearance of the original object in unseen views while maintaining consistency in spatial positioning, observed geometry, and appearance. Across challenging benchmarks, SCORP achieves consistent gains over recent state-of-the-art baselines on both novel view synthesis and geometry completion tasks. SCORP is available at https://github.com/PolySummit/SCORP.