Deep Deformable 3D Caricatures with Learned Shape Control

TL;DR

This paper introduces a data-driven, neural network-based framework for modeling and editing 3D caricatures, enabling exaggerated and natural shape variations through learned shape control.

Contribution

It presents a novel MLP-based deformable surface model with a hypernetwork for compact caricature shape representation and editing capabilities.

Findings

Supports semantic and handle-based editing of 3D caricatures

Enables automatic creation of 3D caricatures

Produces highly exaggerated, natural shapes

Abstract

A 3D caricature is an exaggerated 3D depiction of a human face. The goal of this paper is to model the variations of 3D caricatures in a compact parameter space so that we can provide a useful data-driven toolkit for handling 3D caricature deformations. To achieve the goal, we propose an MLP-based framework for building a deformable surface model, which takes a latent code and produces a 3D surface. In the framework, a SIREN MLP models a function that takes a 3D position on a fixed template surface and returns a 3D displacement vector for the input position. We create variations of 3D surfaces by learning a hypernetwork that takes a latent code and produces the parameters of the MLP. Once learned, our deformable model provides a nice editing space for 3D caricatures, supporting label-based semantic editing and point-handle-based deformation, both of which produce highly exaggerated and natural 3D caricature shapes. We also demonstrate other applications of our deformable model, such as automatic 3D caricature creation.