Unsupervised Doodling and Painting with Improved SPIRAL

TL;DR

This paper explores reinforcement learning agents as generative models for images, demonstrating they can produce abstract and realistic images through improved architectures without external supervision.

Contribution

It introduces architectural improvements to RL-based generative agents, enabling the emergence of visual abstraction and realism in image creation from photographs.

Findings

Agents can generate abstract images despite only seeing real photos.

With sufficient training, agents produce images with high realism.

The framework has potential applications in creative image generation.

Abstract

We investigate using reinforcement learning agents as generative models of images (extending arXiv:1804.01118). A generative agent controls a simulated painting environment, and is trained with rewards provided by a discriminator network simultaneously trained to assess the realism of the agent's samples, either unconditional or reconstructions. Compared to prior work, we make a number of improvements to the architectures of the agents and discriminators that lead to intriguing and at times surprising results. We find that when sufficiently constrained, generative agents can learn to produce images with a degree of visual abstraction, despite having only ever seen real photographs (no human brush strokes). And given enough time with the painting environment, they can produce images with considerable realism. These results show that, under the right circumstances, some aspects of human drawing can emerge from simulated embodiment, without the need for external supervision, imitation or social cues. Finally, we note the framework's potential for use in creative applications.