SketchMetaFace: A Learning-based Sketching Interface for High-fidelity 3D Character Face Modeling

TL;DR

SketchMetaFace is a user-friendly sketching system that enables amateurs to quickly create high-fidelity 3D character faces using a novel learning-based modeling method and an intuitive interface.

Contribution

The paper introduces SketchMetaFace, a new sketching system with a novel IDGMM algorithm for efficient, high-quality 3D face modeling suitable for non-expert users.

Findings

User studies show improved ease of use and visual quality.

IDGMM achieves a better accuracy-efficiency trade-off.

System significantly reduces modeling time for amateurs.

Abstract

Modeling 3D avatars benefits various application scenarios such as AR/VR, gaming, and filming. Character faces contribute significant diversity and vividity as a vital component of avatars. However, building 3D character face models usually requires a heavy workload with commercial tools, even for experienced artists. Various existing sketch-based tools fail to support amateurs in modeling diverse facial shapes and rich geometric details. In this paper, we present SketchMetaFace - a sketching system targeting amateur users to model high-fidelity 3D faces in minutes. We carefully design both the user interface and the underlying algorithm. First, curvature-aware strokes are adopted to better support the controllability of carving facial details. Second, considering the key problem of mapping a 2D sketch map to a 3D model, we develop a novel learning-based method termed "Implicit and Depth Guided Mesh Modeling" (IDGMM). It fuses the advantages of mesh, implicit, and depth representations to achieve high-quality results with high efficiency. In addition, to further support usability, we present a coarse-to-fine 2D sketching interface design and a data-driven stroke suggestion tool. User studies demonstrate the superiority of our system over existing modeling tools in terms of the ease to use and visual quality of results. Experimental analyses also show that IDGMM reaches a better trade-off between accuracy and efficiency. SketchMetaFace is available at https://zhongjinluo.github.io/SketchMetaFace/.