Skel3D: Skeleton Guided Novel View Synthesis
Aron F\'othi, Bence Fazekas, Natabara M\'at\'e Gy\"ongy\"ossy,, Kristian Fenech

TL;DR
Skel3D introduces a skeleton-guided diffusion approach for monocular novel view synthesis that improves pose accuracy and multi-view consistency by leveraging object skeletons, outperforming existing methods without explicit 3D models.
Contribution
The paper presents a novel skeleton-guided diffusion method that enhances view synthesis quality by incorporating structural skeleton information into the generative process.
Findings
Significantly improves pose accuracy and multi-view consistency.
Outperforms state-of-the-art NVS methods quantitatively and qualitatively.
Effectively leverages object skeletons without explicit 3D representations.
Abstract
In this paper, we present an approach for monocular open-set novel view synthesis (NVS) that leverages object skeletons to guide the underlying diffusion model. Building upon a baseline that utilizes a pre-trained 2D image generator, our method takes advantage of the Objaverse dataset, which includes animated objects with bone structures. By introducing a skeleton guide layer following the existing ray conditioning normalization (RCN) layer, our approach enhances pose accuracy and multi-view consistency. The skeleton guide layer provides detailed structural information for the generative model, improving the quality of synthesized views. Experimental results demonstrate that our skeleton-guided method significantly enhances consistency and accuracy across diverse object categories within the Objaverse dataset. Our method outperforms existing state-of-the-art NVS techniques both…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsGeological Modeling and Analysis · Computer Graphics and Visualization Techniques · Simulation and Modeling Applications
MethodsDiffusion
