SINGAPO: Single Image Controlled Generation of Articulated Parts in Objects

TL;DR

SINGAPO introduces a novel single-image method for generating articulated 3D objects, utilizing a diffusion model and a structured coarse-to-fine pipeline to produce realistic, consistent, and controllable models from minimal input.

Contribution

The paper presents a new single-image approach for articulated object creation that combines a diffusion model with a structured coarse-to-fine pipeline, overcoming previous multi-view and coarse control limitations.

Findings

Outperforms state-of-the-art in realism and resemblance

Generates consistent articulated objects from a single image

Achieves high-quality reconstruction of geometric details

Abstract

We address the challenge of creating 3D assets for household articulated objects from a single image. Prior work on articulated object creation either requires multi-view multi-state input, or only allows coarse control over the generation process. These limitations hinder the scalability and practicality for articulated object modeling. In this work, we propose a method to generate articulated objects from a single image. Observing the object in resting state from an arbitrary view, our method generates an articulated object that is visually consistent with the input image. To capture the ambiguity in part shape and motion posed by a single view of the object, we design a diffusion model that learns the plausible variations of objects in terms of geometry and kinematics. To tackle the complexity of generating structured data with attributes in multiple domains, we design a pipeline that produces articulated objects from high-level structure to geometric details in a coarse-to-fine manner, where we use a part connectivity graph and part abstraction as proxies. Our experiments show that our method outperforms the state-of-the-art in articulated object creation by a large margin in terms of the generated object realism, resemblance to the input image, and reconstruction quality.