CloseUpShot: Close-up Novel View Synthesis from Sparse-views via Point-conditioned Diffusion Model

TL;DR

CloseUpShot introduces a diffusion-based method for high-quality close-up novel view synthesis from sparse views, utilizing hierarchical warping, occlusion-aware noise suppression, and global structure guidance to improve detail and consistency.

Contribution

The paper proposes a novel point-conditioned diffusion framework with hierarchical warping and global structure guidance for close-up view synthesis from sparse inputs.

Findings

Outperforms existing methods in close-up view synthesis

Enhances detail and completeness in sparse-view scenarios

Effectively leverages dense point clouds for geometric consistency

Abstract

Reconstructing 3D scenes and synthesizing novel views from sparse input views is a highly challenging task. Recent advances in video diffusion models have demonstrated strong temporal reasoning capabilities, making them a promising tool for enhancing reconstruction quality under sparse-view settings. However, existing approaches are primarily designed for modest viewpoint variations, which struggle in capturing fine-grained details in close-up scenarios since input information is severely limited. In this paper, we present a diffusion-based framework, called CloseUpShot, for close-up novel view synthesis from sparse inputs via point-conditioned video diffusion. Specifically, we observe that pixel-warping conditioning suffers from severe sparsity and background leakage in close-up settings. To address this, we propose hierarchical warping and occlusion-aware noise suppression, enhancing the quality and completeness of the conditioning images for the video diffusion model. Furthermore, we introduce global structure guidance, which leverages a dense fused point cloud to provide consistent geometric context to the diffusion process, to compensate for the lack of globally consistent 3D constraints in sparse conditioning inputs. Extensive experiments on multiple datasets demonstrate that our method outperforms existing approaches, especially in close-up novel view synthesis, clearly validating the effectiveness of our design.