Locally-Focused Face Representation for Sketch-to-Image Generation Using Noise-Induced Refinement

TL;DR

This paper introduces a deep-learning framework using attention mechanisms and noise-induced cGANs to convert face sketches into high-quality color images, outperforming existing methods in realism and fidelity.

Contribution

The novel CA2N architecture with block attention and a noise-induced cGAN process improves sketch-to-image generation, achieving state-of-the-art results and cross-domain generalization.

Findings

Outperforms previous methods with significant FID improvements on multiple datasets.

Achieves high image realism and fidelity in sketch-to-image translation.

Demonstrates robustness and generalization across unseen sketch domains.

Abstract

This paper presents a novel deep-learning framework that significantly enhances the transformation of rudimentary face sketches into high-fidelity colour images. Employing a Convolutional Block Attention-based Auto-encoder Network (CA2N), our approach effectively captures and enhances critical facial features through a block attention mechanism within an encoder-decoder architecture. Subsequently, the framework utilises a noise-induced conditional Generative Adversarial Network (cGAN) process that allows the system to maintain high performance even on domains unseen during the training. These enhancements lead to considerable improvements in image realism and fidelity, with our model achieving superior performance metrics that outperform the best method by FID margin of 17, 23, and 38 on CelebAMask-HQ, CUHK, and CUFSF datasets; respectively. The model sets a new state-of-the-art in sketch-to-image generation, can generalize across sketch types, and offers a robust solution for applications such as criminal identification in law enforcement.