LinesToFacePhoto: Face Photo Generation from Lines with Conditional Self-Attention Generative Adversarial Network

TL;DR

This paper introduces CSAGAN, a novel model that generates realistic face images from line drawings by capturing long-range dependencies and ensuring structural completeness, outperforming existing methods.

Contribution

We propose a conditional self-attention mechanism within cGANs and a multi-scale discriminator to improve face image synthesis from line inputs.

Findings

Outperforms state-of-the-art methods quantitatively.

Produces high-quality, structurally consistent face images.

Validated by user studies and multiple metrics.

Abstract

In this paper, we explore the task of generating photo-realistic face images from lines. Previous methods based on conditional generative adversarial networks (cGANs) have shown their power to generate visually plausible images when a conditional image and an output image share well-aligned structures. However, these models fail to synthesize face images with a whole set of well-defined structures, e.g. eyes, noses, mouths, etc., especially when the conditional line map lacks one or several parts. To address this problem, we propose a conditional self-attention generative adversarial network (CSAGAN). We introduce a conditional self-attention mechanism to cGANs to capture long-range dependencies between different regions in faces. We also build a multi-scale discriminator. The large-scale discriminator enforces the completeness of global structures and the small-scale discriminator encourages fine details, thereby enhancing the realism of generated face images. We evaluate the proposed model on the CelebA-HD dataset by two perceptual user studies and three quantitative metrics. The experiment results demonstrate that our method generates high-quality facial images while preserving facial structures. Our results outperform state-of-the-art methods both quantitatively and qualitatively.