Dr.3D: Adapting 3D GANs to Artistic Drawings

TL;DR

This paper introduces Dr.3D, a novel method that adapts 3D GANs to generate multi-view consistent artistic portrait drawings, overcoming geometric ambiguity through specialized components.

Contribution

It presents a new adaptation approach with three components to extend 3D GANs from realistic images to artistic drawings, addressing geometric ambiguity.

Findings

Successfully adapts 3D GANs to artistic drawings

Enables multi-view consistent semantic editing of drawings

Handles geometric ambiguity with novel components

Abstract

While 3D GANs have recently demonstrated the high-quality synthesis of multi-view consistent images and 3D shapes, they are mainly restricted to photo-realistic human portraits. This paper aims to extend 3D GANs to a different, but meaningful visual form: artistic portrait drawings. However, extending existing 3D GANs to drawings is challenging due to the inevitable geometric ambiguity present in drawings. To tackle this, we present Dr.3D, a novel adaptation approach that adapts an existing 3D GAN to artistic drawings. Dr.3D is equipped with three novel components to handle the geometric ambiguity: a deformation-aware 3D synthesis network, an alternating adaptation of pose estimation and image synthesis, and geometric priors. Experiments show that our approach can successfully adapt 3D GANs to drawings and enable multi-view consistent semantic editing of drawings.