MegaPortraits: One-shot Megapixel Neural Head Avatars

TL;DR

This paper introduces a novel neural head avatar system capable of generating high-resolution, realistic avatars with cross-driving synthesis, outperforming existing methods and enabling real-time, identity-locked applications.

Contribution

It presents new neural architectures and training techniques that leverage both medium and high-resolution data for high-quality, generalizable neural head avatars at megapixel resolution.

Findings

Achieved convincing high-resolution neural avatars.

Outperformed competitors in cross-driving scenarios.

Enabled real-time, identity-locked avatar operation.

Abstract

In this work, we advance the neural head avatar technology to the megapixel resolution while focusing on the particularly challenging task of cross-driving synthesis, i.e., when the appearance of the driving image is substantially different from the animated source image. We propose a set of new neural architectures and training methods that can leverage both medium-resolution video data and high-resolution image data to achieve the desired levels of rendered image quality and generalization to novel views and motion. We demonstrate that suggested architectures and methods produce convincing high-resolution neural avatars, outperforming the competitors in the cross-driving scenario. Lastly, we show how a trained high-resolution neural avatar model can be distilled into a lightweight student model which runs in real-time and locks the identities of neural avatars to several dozens of pre-defined source images. Real-time operation and identity lock are essential for many practical applications head avatar systems.