Learning Implicit Representation for Reconstructing Articulated Objects

TL;DR

This paper presents a category-agnostic method for 3D reconstruction of articulated objects from video, estimating both explicit shapes and implicit skeletal structures without prior 3D supervision.

Contribution

It introduces a novel approach that jointly estimates explicit 3D shapes and implicit skeletal representations from motion cues, generalizing beyond category-specific models.

Findings

Outperforms state-of-the-art methods on standard datasets

Eliminates need for category-specific skeletal models

Successfully reconstructs articulated objects in the wild

Abstract

3D Reconstruction of moving articulated objects without additional information about object structure is a challenging problem. Current methods overcome such challenges by employing category-specific skeletal models. Consequently, they do not generalize well to articulated objects in the wild. We treat an articulated object as an unknown, semi-rigid skeletal structure surrounded by nonrigid material (e.g., skin). Our method simultaneously estimates the visible (explicit) representation (3D shapes, colors, camera parameters) and the implicit skeletal representation, from motion cues in the object video without 3D supervision. Our implicit representation consists of four parts. (1) Skeleton, which specifies how semi-rigid parts are connected. (2) \textcolor{black}{Skinning Weights}, which associates each surface vertex with semi-rigid parts with probability. (3) Rigidity Coefficients, specifying the articulation of the local surface. (4) Time-Varying Transformations, which specify the skeletal motion and surface deformation parameters. We introduce an algorithm that uses physical constraints as regularization terms and iteratively estimates both implicit and explicit representations. Our method is category-agnostic, thus eliminating the need for category-specific skeletons, we show that our method outperforms state-of-the-art across standard video datasets.