High-Fidelity Clothed Avatar Reconstruction from a Single Image

TL;DR

This paper introduces a hybrid framework for reconstructing high-fidelity 3D clothed human avatars from a single image, combining learning-based shape estimation with optimization-based surface refinement.

Contribution

It proposes a coarse-to-fine approach using an implicit model and non-rigid deformation, with a hyper-network for fast initialization, advancing single-image avatar reconstruction.

Findings

Successfully produces high-fidelity avatars for clothed humans

Accelerates optimization convergence with hyper-network initialization

Demonstrates effectiveness on various real scene datasets

Abstract

This paper presents a framework for efficient 3D clothed avatar reconstruction. By combining the advantages of the high accuracy of optimization-based methods and the efficiency of learning-based methods, we propose a coarse-to-fine way to realize a high-fidelity clothed avatar reconstruction (CAR) from a single image. At the first stage, we use an implicit model to learn the general shape in the canonical space of a person in a learning-based way, and at the second stage, we refine the surface detail by estimating the non-rigid deformation in the posed space in an optimization way. A hyper-network is utilized to generate a good initialization so that the convergence o f the optimization process is greatly accelerated. Extensive experiments on various datasets show that the proposed CAR successfully produces high-fidelity avatars for arbitrarily clothed humans in real scenes.