Semantic Style Transfer and Turning Two-Bit Doodles into Fine Artworks
Alex J. Champandard

TL;DR
This paper presents a semantic-aware generative approach that enhances style transfer and transforms simple doodles into detailed artworks by incorporating semantic annotations for better control and quality.
Contribution
It introduces a novel method to augment CNN-based image synthesis with semantic labels, improving control, quality, and applicability of style transfer and doodle-to-art transformations.
Findings
Improved image quality and plausibility in style transfer.
Enhanced control over generated images through semantic annotations.
Successful transformation of simple doodles into detailed artworks.
Abstract
Convolutional neural networks (CNNs) have proven highly effective at image synthesis and style transfer. For most users, however, using them as tools can be a challenging task due to their unpredictable behavior that goes against common intuitions. This paper introduces a novel concept to augment such generative architectures with semantic annotations, either by manually authoring pixel labels or using existing solutions for semantic segmentation. The result is a content-aware generative algorithm that offers meaningful control over the outcome. Thus, we increase the quality of images generated by avoiding common glitches, make the results look significantly more plausible, and extend the functional range of these algorithms---whether for portraits or landscapes, etc. Applications include semantic style transfer and turning doodles with few colors into masterful paintings!
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
From Doodles To Paintings With Deep Learning | Two Minute Papers #57· youtube
Taxonomy
TopicsGenerative Adversarial Networks and Image Synthesis · Computer Graphics and Visualization Techniques · Aesthetic Perception and Analysis
