Continuation of Famous Art with AI: A Conditional Adversarial Network Inpainting Approach

TL;DR

This paper presents a conditional GAN-based inpainting method to extend and generate famous artworks, capturing textures, scenery, and details like text, demonstrated across diverse art datasets.

Contribution

It introduces a novel inpainting approach using a conditional GAN trained on cropped images to generate extended artwork continuations, including texture and scene details.

Findings

Successfully extended artworks with realistic textures and scenery.

Generated features like sky, water, and text even in border regions.

Performed well across diverse art styles and datasets.

Abstract

Much of the state-of-the-art in image synthesis inspired by real artwork are either entirely generative by filtered random noise or inspired by the transfer of style. This work explores the application of image inpainting to continue famous artworks and produce generative art with a Conditional GAN. During the training stage of the process, the borders of images are cropped, leaving only the centre. An inpainting GAN is then tasked with learning to reconstruct the original image from the centre crop by way of minimising both adversarial and absolute difference losses, which are analysed by both their Fr\'echet Inception Distances and manual observations which are presented. Once the network is trained, images are then resized rather than cropped and presented as input to the generator. Following the learning process, the generator then creates new images by continuing from the edges of the original piece. Three experiments are performed with datasets of 4766 landscape paintings (impressionism and romanticism), 1167 Ukiyo-e works from the Japanese Edo period, and 4968 abstract artworks. Results show that geometry and texture (including canvas and paint) as well as scenery such as sky, clouds, water, land (including hills and mountains), grass, and flowers are implemented by the generator when extending real artworks. In the Ukiyo-e experiments, it was observed that features such as written text were generated even in cases where the original image did not have any, due to the presence of an unpainted border within the input image.