NECA: Neural Customizable Human Avatar

TL;DR

NECA introduces a versatile neural human avatar model capable of detailed customization and realistic rendering from monocular or sparse-view videos, advancing photorealistic avatar creation and editing.

Contribution

We propose a dual-space neural representation for human avatars enabling detailed customization and high-quality rendering from limited video data.

Findings

Outperforms state-of-the-art in photorealistic rendering

Enables detailed avatar editing such as pose and relighting

Demonstrates high-frequency detail preservation in rendering

Abstract

Human avatar has become a novel type of 3D asset with various applications. Ideally, a human avatar should be fully customizable to accommodate different settings and environments. In this work, we introduce NECA, an approach capable of learning versatile human representation from monocular or sparse-view videos, enabling granular customization across aspects such as pose, shadow, shape, lighting and texture. The core of our approach is to represent humans in complementary dual spaces and predict disentangled neural fields of geometry, albedo, shadow, as well as an external lighting, from which we are able to derive realistic rendering with high-frequency details via volumetric rendering. Extensive experiments demonstrate the advantage of our method over the state-of-the-art methods in photorealistic rendering, as well as various editing tasks such as novel pose synthesis and relighting. The code is available at https://github.com/iSEE-Laboratory/NECA.