Object Wake-up: 3D Object Rigging from a Single Image

TL;DR

This paper introduces an automated method to reconstruct 3D objects from a single image and embed articulated skeletons, enabling plausible animations for generic objects with applications in AR and VR.

Contribution

It presents a novel framework combining 3D reconstruction and skeleton prediction for generic objects from a single image, surpassing previous methods in accuracy.

Findings

Outperforms state-of-the-art on 3D reconstruction tasks

Accurately predicts skeletons for diverse objects

Demonstrates applicability in AR/VR scenarios

Abstract

Given a single image of a general object such as a chair, could we also restore its articulated 3D shape similar to human modeling, so as to animate its plausible articulations and diverse motions? This is an interesting new question that may have numerous downstream augmented reality and virtual reality applications. Comparing with previous efforts on object manipulation, our work goes beyond 2D manipulation and rigid deformation, and involves articulated manipulation. To achieve this goal, we propose an automated approach to build such 3D generic objects from single images and embed articulated skeletons in them. Specifically, our framework starts by reconstructing the 3D object from an input image. Afterwards, to extract skeletons for generic 3D objects, we develop a novel skeleton prediction method with a multi-head structure for skeleton probability field estimation by utilizing the deep implicit functions. A dataset of generic 3D objects with ground-truth annotated skeletons is collected. Empirically our approach is demonstrated with satisfactory performance on public datasets as well as our in-house dataset; our results surpass those of the state-of-the-arts by a noticeable margin on both 3D reconstruction and skeleton prediction.