RLMiniStyler: Light-weight RL Style Agent for Arbitrary Sequential Neural Style Generation

TL;DR

RLMiniStyler is a lightweight reinforcement learning framework that efficiently generates diverse artistic stylized image sequences, balancing quality and computational cost.

Contribution

It introduces a novel RL-based approach with an uncertainty-aware multi-task learning strategy for efficient arbitrary style transfer.

Findings

Outperforms state-of-the-art methods in stylization quality

Generates diverse stylized sequences with lower computational cost

Accelerates model convergence through adaptive loss weighting

Abstract

Arbitrary style transfer aims to apply the style of any given artistic image to another content image. Still, existing deep learning-based methods often require significant computational costs to generate diverse stylized results. Motivated by this, we propose a novel reinforcement learning-based framework for arbitrary style transfer RLMiniStyler. This framework leverages a unified reinforcement learning policy to iteratively guide the style transfer process by exploring and exploiting stylization feedback, generating smooth sequences of stylized results while achieving model lightweight. Furthermore, we introduce an uncertainty-aware multi-task learning strategy that automatically adjusts loss weights to adapt to the content and style balance requirements at different training stages, thereby accelerating model convergence. Through a series of experiments across image various resolutions, we have validated the advantages of RLMiniStyler over other state-of-the-art methods in generating high-quality, diverse artistic image sequences at a lower cost. Codes are available at https://github.com/fengxiaoming520/RLMiniStyler.