Low-Rank Head Avatar Personalization with Registers

TL;DR

This paper presents a new low-rank personalization method for head avatar generation that uses a Register Module to better capture identity-specific facial details with minimal parameters.

Contribution

The introduction of a Register Module that enhances low-rank adaptation for personalized head avatars, requiring fewer parameters and better capturing high-frequency facial details.

Findings

Outperforms existing personalization methods quantitatively.

Faithfully captures distinctive facial details like wrinkles and tattoos.

Requires only a small number of parameters for adaptation.

Abstract

We introduce a novel method for low-rank personalization of a generic model for head avatar generation. Prior work proposes generic models that achieve high-quality face animation by leveraging large-scale datasets of multiple identities. However, such generic models usually fail to synthesize unique identity-specific details, since they learn a general domain prior. To adapt to specific subjects, we find that it is still challenging to capture high-frequency facial details via popular solutions like low-rank adaptation (LoRA). This motivates us to propose a specific architecture, a Register Module, that enhances the performance of LoRA, while requiring only a small number of parameters to adapt to an unseen identity. Our module is applied to intermediate features of a pre-trained model, storing and re-purposing information in a learnable 3D feature space. To demonstrate the efficacy of our personalization method, we collect a dataset of talking videos of individuals with distinctive facial details, such as wrinkles and tattoos. Our approach faithfully captures unseen faces, outperforming existing methods quantitatively and qualitatively. We will release the code, models, and dataset to the public.