iHuman: Instant Animatable Digital Humans From Monocular Videos

TL;DR

This paper introduces a fast and effective method for creating animatable 3D digital humans from monocular videos, leveraging Gaussian splatting and novel modeling techniques to improve accuracy and speed.

Contribution

It presents a new pipeline that combines implicit surface modeling, Gaussian binding to mesh faces, and normal rendering supervision for improved 3D human digitization.

Findings

Achieves state-of-the-art results on benchmark datasets.

Training time is an order of magnitude faster than previous methods.

Produces superior rendering and 3D reconstruction across different poses.

Abstract

Personalized 3D avatars require an animatable representation of digital humans. Doing so instantly from monocular videos offers scalability to broad class of users and wide-scale applications. In this paper, we present a fast, simple, yet effective method for creating animatable 3D digital humans from monocular videos. Our method utilizes the efficiency of Gaussian splatting to model both 3D geometry and appearance. However, we observed that naively optimizing Gaussian splats results in inaccurate geometry, thereby leading to poor animations. This work achieves and illustrates the need of accurate 3D mesh-type modelling of the human body for animatable digitization through Gaussian splats. This is achieved by developing a novel pipeline that benefits from three key aspects: (a) implicit modelling of surface's displacements and the color's spherical harmonics; (b) binding of 3D Gaussians to the respective triangular faces of the body template; (c) a novel technique to render normals followed by their auxiliary supervision. Our exhaustive experiments on three different benchmark datasets demonstrates the state-of-the-art results of our method, in limited time settings. In fact, our method is faster by an order of magnitude (in terms of training time) than its closest competitor. At the same time, we achieve superior rendering and 3D reconstruction performance under the change of poses.