Content Masked Loss: Human-Like Brush Stroke Planning in a Reinforcement Learning Painting Agent

TL;DR

This paper introduces Content Masked Loss, a novel reward function for reinforcement learning painting agents that emphasizes important image features, resulting in more human-like stroke planning and earlier subject recognition in generated artworks.

Contribution

The paper proposes Content Masked Loss, a new loss function that guides RL painting agents to produce more human-like strokes without needing costly human data.

Findings

Paintings with Content Masked Loss show earlier subject recognition.

The method improves human-likeness of stroke sequences.

Final painting quality remains high.

Abstract

The objective of most Reinforcement Learning painting agents is to minimize the loss between a target image and the paint canvas. Human painter artistry emphasizes important features of the target image rather than simply reproducing it (DiPaola 2007). Using adversarial or L2 losses in the RL painting models, although its final output is generally a work of finesse, produces a stroke sequence that is vastly different from that which a human would produce since the model does not have knowledge about the abstract features in the target image. In order to increase the human-like planning of the model without the use of expensive human data, we introduce a new loss function for use with the model's reward function: Content Masked Loss. In the context of robot painting, Content Masked Loss employs an object detection model to extract features which are used to assign higher weight to regions of the canvas that a human would find important for recognizing content. The results, based on 332 human evaluators, show that the digital paintings produced by our Content Masked model show detectable subject matter earlier in the stroke sequence than existing methods without compromising on the quality of the final painting. Our code is available at https://github.com/pschaldenbrand/ContentMaskedLoss.