Efficient Neural Implicit Representation for 3D Human Reconstruction

TL;DR

HumanAvatar is a novel method that efficiently reconstructs high-quality 3D human avatars from monocular videos by integrating advanced models and techniques, achieving faster training and superior accuracy compared to existing methods.

Contribution

The paper introduces HumanAvatar, combining HuMoR, Instant-NGP, and Fast-SNARF, with a posture-sensitive space reduction to significantly improve speed and quality of 3D human reconstruction from monocular videos.

Findings

Achieves 110X faster training than state-of-the-art NeRF models.

Provides comparable or better quality reconstructions in minutes.

Effective visual results after only 30 seconds of training.

Abstract

High-fidelity digital human representations are increasingly in demand in the digital world, particularly for interactive telepresence, AR/VR, 3D graphics, and the rapidly evolving metaverse. Even though they work well in small spaces, conventional methods for reconstructing 3D human motion frequently require the use of expensive hardware and have high processing costs. This study presents HumanAvatar, an innovative approach that efficiently reconstructs precise human avatars from monocular video sources. At the core of our methodology, we integrate the pre-trained HuMoR, a model celebrated for its proficiency in human motion estimation. This is adeptly fused with the cutting-edge neural radiance field technology, Instant-NGP, and the state-of-the-art articulated model, Fast-SNARF, to enhance the reconstruction fidelity and speed. By combining these two technologies, a system is created that can render quickly and effectively while also providing estimation of human pose parameters that are unmatched in accuracy. We have enhanced our system with an advanced posture-sensitive space reduction technique, which optimally balances rendering quality with computational efficiency. In our detailed experimental analysis using both artificial and real-world monocular videos, we establish the advanced performance of our approach. HumanAvatar consistently equals or surpasses contemporary leading-edge reconstruction techniques in quality. Furthermore, it achieves these complex reconstructions in minutes, a fraction of the time typically required by existing methods. Our models achieve a training speed that is 110X faster than that of State-of-The-Art (SoTA) NeRF-based models. Our technique performs noticeably better than SoTA dynamic human NeRF methods if given an identical runtime limit. HumanAvatar can provide effective visuals after only 30 seconds of training.