ToMiE: Towards Explicit Exoskeleton for the Reconstruction of Complicated 3D Human Avatars

TL;DR

This paper introduces ToMiE, a method that adaptively expands the SMPL skeleton to better model complex 3D human avatars with objects and clothing, improving rendering and animation capabilities.

Contribution

We propose a growth strategy for SMPL skeletons, including parent joint localization and external joint optimization, to handle complex human modeling scenarios.

Findings

Outperforms existing methods in rendering quality.

Enables free animation of expanded joints.

Effectively models humans with objects and loose clothing.

Abstract

In this paper, we highlight a critical yet often overlooked factor in most 3D human tasks, namely modeling complicated 3D human with with hand-held objects or loose-fitting clothing. It is known that the parameterized formulation of SMPL is able to fit human skin; while hand-held objects and loose-fitting clothing, are difficult to get modeled within the unified framework, since their movements are usually decoupled with the human body. To enhance the capability of SMPL skeleton in response to this situation, we propose a growth strategy that enables the joint tree of the skeleton to expand adaptively. Specifically, our method, called ToMiE, consists of parent joints localization and external joints optimization. For parent joints localization, we employ a gradient-based approach guided by both LBS blending weights and motion kernels. Once the external joints are obtained, we proceed to optimize their transformations in SE(3) across different frames, enabling rendering and explicit animation. ToMiE manages to outperform other methods across various cases with hand-held objects and loose-fitting clothing, not only in rendering quality but also by offering free animation of grown joints, thereby enhancing the expressive ability of SMPL skeleton for a broader range of applications.