MultiGO: Towards Multi-level Geometry Learning for Monocular 3D Textured Human Reconstruction

TL;DR

This paper presents MultiGO, a multi-level geometry learning framework that enhances monocular 3D human reconstruction by capturing detailed geometric features, leading to more accurate skeleton, joint, and wrinkle modeling.

Contribution

The paper introduces a novel multi-level geometry learning framework with three key modules for detailed 3D human reconstruction from monocular images.

Findings

Outperforms state-of-the-art methods on two test sets.

Effectively captures geometric details like wrinkles.

Improves joint depth estimation accuracy.

Abstract

This paper investigates the research task of reconstructing the 3D clothed human body from a monocular image. Due to the inherent ambiguity of single-view input, existing approaches leverage pre-trained SMPL(-X) estimation models or generative models to provide auxiliary information for human reconstruction. However, these methods capture only the general human body geometry and overlook specific geometric details, leading to inaccurate skeleton reconstruction, incorrect joint positions, and unclear cloth wrinkles. In response to these issues, we propose a multi-level geometry learning framework. Technically, we design three key components: skeleton-level enhancement, joint-level augmentation, and wrinkle-level refinement modules. Specifically, we effectively integrate the projected 3D Fourier features into a Gaussian reconstruction model, introduce perturbations to improve joint depth estimation during training, and refine the human coarse wrinkles by resembling the de-noising process of diffusion model. Extensive quantitative and qualitative experiments on two out-of-distribution test sets show the superior performance of our approach compared to state-of-the-art (SOTA) methods.