Controlling Neural Style Transfer with Deep Reinforcement Learning

TL;DR

This paper introduces a deep reinforcement learning approach for neural style transfer that allows step-wise control over stylization, preserving content details early on and adding style later, with lower computational costs.

Contribution

It presents the first RL-based architecture for progressive style transfer, enabling user control and improved detail preservation compared to traditional methods.

Findings

Preserves more content details in early steps.

Synthesizes richer style patterns in later steps.

Achieves lower computational complexity.

Abstract

Controlling the degree of stylization in the Neural Style Transfer (NST) is a little tricky since it usually needs hand-engineering on hyper-parameters. In this paper, we propose the first deep Reinforcement Learning (RL) based architecture that splits one-step style transfer into a step-wise process for the NST task. Our RL-based method tends to preserve more details and structures of the content image in early steps, and synthesize more style patterns in later steps. It is a user-easily-controlled style-transfer method. Additionally, as our RL-based model performs the stylization progressively, it is lightweight and has lower computational complexity than existing one-step Deep Learning (DL) based models. Experimental results demonstrate the effectiveness and robustness of our method.