SPAMs: Structured Implicit Parametric Models

TL;DR

SPAMs introduces a structured implicit parametric modeling approach for deformable objects, enabling part-based shape and pose representation that improves reconstruction and tracking of complex motions.

Contribution

The paper proposes a novel part-based implicit model for deformable objects that structurally decomposes motion into shape and pose, enhancing robustness and interpretability.

Findings

Achieves state-of-the-art results in depth sequence reconstruction.

Enables robust joint optimization for shape and pose estimation.

Effectively handles dramatic object motions.

Abstract

Parametric 3D models have formed a fundamental role in modeling deformable objects, such as human bodies, faces, and hands; however, the construction of such parametric models requires significant manual intervention and domain expertise. Recently, neural implicit 3D representations have shown great expressibility in capturing 3D shape geometry. We observe that deformable object motion is often semantically structured, and thus propose to learn Structured-implicit PArametric Models (SPAMs) as a deformable object representation that structurally decomposes non-rigid object motion into part-based disentangled representations of shape and pose, with each being represented by deep implicit functions. This enables a structured characterization of object movement, with part decomposition characterizing a lower-dimensional space in which we can establish coarse motion correspondence. In particular, we can leverage the part decompositions at test time to fit to new depth sequences of unobserved shapes, by establishing part correspondences between the input observation and our learned part spaces; this guides a robust joint optimization between the shape and pose of all parts, even under dramatic motion sequences. Experiments demonstrate that our part-aware shape and pose understanding lead to state-of-the-art performance in reconstruction and tracking of depth sequences of complex deforming object motion. We plan to release models to the public at https://pablopalafox.github.io/spams.