Fast Bi-layer Neural Synthesis of One-Shot Realistic Head Avatars

TL;DR

This paper introduces a neural rendering system that efficiently generates realistic 3D head avatars from a single photo by decomposing appearance into coarse and detailed layers, enabling real-time applications.

Contribution

It presents a novel bi-layer neural synthesis approach that significantly improves inference speed while maintaining high visual quality for head avatar creation from minimal input.

Findings

Achieves faster inference compared to state-of-the-art methods.

Maintains high visual quality in synthesized head avatars.

Enables real-time implementation on smartphones.

Abstract

We propose a neural rendering-based system that creates head avatars from a single photograph. Our approach models a person's appearance by decomposing it into two layers. The first layer is a pose-dependent coarse image that is synthesized by a small neural network. The second layer is defined by a pose-independent texture image that contains high-frequency details. The texture image is generated offline, warped and added to the coarse image to ensure a high effective resolution of synthesized head views. We compare our system to analogous state-of-the-art systems in terms of visual quality and speed. The experiments show significant inference speedup over previous neural head avatar models for a given visual quality. We also report on a real-time smartphone-based implementation of our system.