Neural Image-based Avatars: Generalizable Radiance Fields for Human Avatar Modeling

TL;DR

This paper introduces Neural Image-based Avatars (NIA), a hybrid approach combining implicit body NeRF and image-based rendering to synthesize high-quality novel views and poses of humans from sparse multi-view images, improving generalization and detail preservation.

Contribution

The paper proposes a novel hybrid method that leverages both implicit body NeRF and image-based rendering to enhance human avatar synthesis from sparse views, outperforming existing methods in generalization and quality.

Findings

Outperforms recent methods in identity and cross-dataset generalization.

Achieves superior pose generalization compared to per-subject optimized NeRFs.

Maintains appearance details even with sparse source views.

Abstract

We present a method that enables synthesizing novel views and novel poses of arbitrary human performers from sparse multi-view images. A key ingredient of our method is a hybrid appearance blending module that combines the advantages of the implicit body NeRF representation and image-based rendering. Existing generalizable human NeRF methods that are conditioned on the body model have shown robustness against the geometric variation of arbitrary human performers. Yet they often exhibit blurry results when generalized onto unseen identities. Meanwhile, image-based rendering shows high-quality results when sufficient observations are available, whereas it suffers artifacts in sparse-view settings. We propose Neural Image-based Avatars (NIA) that exploits the best of those two methods: to maintain robustness under new articulations and self-occlusions while directly leveraging the available (sparse) source view colors to preserve appearance details of new subject identities. Our hybrid design outperforms recent methods on both in-domain identity generalization as well as challenging cross-dataset generalization settings. Also, in terms of the pose generalization, our method outperforms even the per-subject optimized animatable NeRF methods. The video results are available at https://youngjoongunc.github.io/nia