DANBO: Disentangled Articulated Neural Body Representations via Graph Neural Networks

TL;DR

DANBO introduces a three-stage graph neural network-based approach to create realistic, disentangled 3D human body models directly from raw images, improving generalization and reducing implausible deformations.

Contribution

The paper proposes a novel three-stage method with explicit body part correlations and localized features, enhancing pose generalization and realism in neural human body representations.

Findings

Produces realistic body shapes under unseen poses

Achieves high-quality image synthesis

Outperforms competing methods in robustness

Abstract

Deep learning greatly improved the realism of animatable human models by learning geometry and appearance from collections of 3D scans, template meshes, and multi-view imagery. High-resolution models enable photo-realistic avatars but at the cost of requiring studio settings not available to end users. Our goal is to create avatars directly from raw images without relying on expensive studio setups and surface tracking. While a few such approaches exist, those have limited generalization capabilities and are prone to learning spurious (chance) correlations between irrelevant body parts, resulting in implausible deformations and missing body parts on unseen poses. We introduce a three-stage method that induces two inductive biases to better disentangled pose-dependent deformation. First, we model correlations of body parts explicitly with a graph neural network. Second, to further reduce the effect of chance correlations, we introduce localized per-bone features that use a factorized volumetric representation and a new aggregation function. We demonstrate that our model produces realistic body shapes under challenging unseen poses and shows high-quality image synthesis. Our proposed representation strikes a better trade-off between model capacity, expressiveness, and robustness than competing methods. Project website: https://lemonatsu.github.io/danbo.