LPMM: Intuitive Pose Control for Neural Talking-Head Model via Landmark-Parameter Morphable Model

TL;DR

This paper introduces LPMM, a landmark-parameter morphable model that enables intuitive, semantic control over head pose and facial expressions in neural talking-head models, improving user-friendliness and precision.

Contribution

The paper proposes a novel landmark-parameter morphable model (LPMM) that allows semantic and intuitive pose control over neural talking-head models, addressing limitations of existing methods.

Findings

LPMM enables precise control of head pose and facial expressions.

The approach maintains photorealism while offering semantic control.

It allows both parameter-based and image-based input adjustments.

Abstract

While current talking head models are capable of generating photorealistic talking head videos, they provide limited pose controllability. Most methods require specific video sequences that should exactly contain the head pose desired, being far from user-friendly pose control. Three-dimensional morphable models (3DMM) offer semantic pose control, but they fail to capture certain expressions. We present a novel method that utilizes parametric control of head orientation and facial expression over a pre-trained neural-talking head model. To enable this, we introduce a landmark-parameter morphable model (LPMM), which offers control over the facial landmark domain through a set of semantic parameters. Using LPMM, it is possible to adjust specific head pose factors, without distorting other facial attributes. The results show our approach provides intuitive rig-like control over neural talking head models, allowing both parameter and image-based inputs.