Neural-Polyptych: Content Controllable Painting Recreation for Diverse Genres
Yiming Zhao, Dewen Guo, Zhouhui Lian, Yue Gao, Jianhong Han, Jie Feng,, Guoping Wang, Bingfeng Zhou, Sheng Li

TL;DR
Neural-Polyptych is a unified GAN-based framework that enables content-controllable, high-resolution digital painting creation by integrating sketches with original artwork fragments, supporting diverse artistic styles and applications.
Contribution
It introduces a multi-scale GAN architecture with a novel Correspondence Attention module and Reference Bank strategy for detailed, global, and local feature integration in high-resolution painting generation.
Findings
Successfully generates megapixel-scale paintings across diverse genres.
Enables applications like large painting extension and genre switching.
Achieves high-fidelity, detailed artwork with global consistency.
Abstract
To bridge the gap between artists and non-specialists, we present a unified framework, Neural-Polyptych, to facilitate the creation of expansive, high-resolution paintings by seamlessly incorporating interactive hand-drawn sketches with fragments from original paintings. We have designed a multi-scale GAN-based architecture to decompose the generation process into two parts, each responsible for identifying global and local features. To enhance the fidelity of semantic details generated from users' sketched outlines, we introduce a Correspondence Attention module utilizing our Reference Bank strategy. This ensures the creation of high-quality, intricately detailed elements within the artwork. The final result is achieved by carefully blending these local elements while preserving coherent global consistency. Consequently, this methodology enables the production of digital paintings at…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAesthetic Perception and Analysis · Image Processing and 3D Reconstruction · Digital Media and Visual Art
MethodsSoftmax · Attention Is All You Need
