Learning to Paint With Model-based Deep Reinforcement Learning

TL;DR

This paper introduces a neural renderer-based model-based deep reinforcement learning approach that enables machines to paint complex images with a small number of strokes, achieving high-quality visual effects without human stroke data.

Contribution

It presents a novel DRL framework with a neural renderer that learns to plan stroke placement and color, effectively decomposing images into strokes without human stroke data.

Findings

High-quality paintings with hundreds of strokes achieved

Training does not require human stroke data

Effective long-term planning for stroke placement

Abstract

We show how to teach machines to paint like human painters, who can use a small number of strokes to create fantastic paintings. By employing a neural renderer in model-based Deep Reinforcement Learning (DRL), our agents learn to determine the position and color of each stroke and make long-term plans to decompose texture-rich images into strokes. Experiments demonstrate that excellent visual effects can be achieved using hundreds of strokes. The training process does not require the experience of human painters or stroke tracking data. The code is available at https://github.com/hzwer/ICCV2019-LearningToPaint.