I M Avatar: Implicit Morphable Head Avatars from Videos

TL;DR

IMavatar is a new method for creating detailed, controllable 3D head avatars from monocular videos, combining the fine control of traditional models with the realism of neural representations.

Contribution

It introduces a novel implicit head avatar model that uses learned blendshapes and skinning fields for expression and pose control, enabling end-to-end training from videos.

Findings

Improves geometry accuracy over state-of-the-art methods.

Provides more complete expression coverage.

Enables controllable, photorealistic head avatars from monocular videos.

Abstract

Traditional 3D morphable face models (3DMMs) provide fine-grained control over expression but cannot easily capture geometric and appearance details. Neural volumetric representations approach photorealism but are hard to animate and do not generalize well to unseen expressions. To tackle this problem, we propose IMavatar (Implicit Morphable avatar), a novel method for learning implicit head avatars from monocular videos. Inspired by the fine-grained control mechanisms afforded by conventional 3DMMs, we represent the expression- and pose- related deformations via learned blendshapes and skinning fields. These attributes are pose-independent and can be used to morph the canonical geometry and texture fields given novel expression and pose parameters. We employ ray marching and iterative root-finding to locate the canonical surface intersection for each pixel. A key contribution is our novel analytical gradient formulation that enables end-to-end training of IMavatars from videos. We show quantitatively and qualitatively that our method improves geometry and covers a more complete expression space compared to state-of-the-art methods.