Hierarchically Structured Neural Bones for Reconstructing Animatable Objects from Casual Videos

TL;DR

This paper introduces a hierarchical deformation model using tree-structured bones to reconstruct and manipulate 3D object models from casual videos, enabling intuitive editing and high-quality results without prior structural knowledge.

Contribution

The paper presents a novel hierarchy deformation framework with a bone occupancy regularization, improving 3D reconstruction and manipulation from casual videos without needing prior object structure.

Findings

Enhanced 3D reconstruction quality from casual videos

Intuitive manipulation of reconstructed models

Effective handling of diverse object instances

Abstract

We propose a new framework for creating and easily manipulating 3D models of arbitrary objects using casually captured videos. Our core ingredient is a novel hierarchy deformation model, which captures motions of objects with a tree-structured bones. Our hierarchy system decomposes motions based on the granularity and reveals the correlations between parts without exploiting any prior structural knowledge. We further propose to regularize the bones to be positioned at the basis of motions, centers of parts, sufficiently covering related surfaces of the part. This is achieved by our bone occupancy function, which identifies whether a given 3D point is placed within the bone. Coupling the proposed components, our framework offers several clear advantages: (1) users can obtain animatable 3D models of the arbitrary objects in improved quality from their casual videos, (2) users can manipulate 3D models in an intuitive manner with minimal costs, and (3) users can interactively add or delete control points as necessary. The experimental results demonstrate the efficacy of our framework on diverse instances, in reconstruction quality, interpretability and easier manipulation. Our code is available at https://github.com/subin6/HSNB.