Style-Agnostic Reinforcement Learning

TL;DR

This paper introduces a style-agnostic reinforcement learning method that uses adversarial style transfer to improve policy generalization across environments with different visual styles, without requiring expert data or labels.

Contribution

It proposes a novel adversarial style perturbation approach integrated into RL training, enabling the learning of invariant representations without additional supervision.

Findings

Outperforms state-of-the-art on Procgen and Distracting Control Suite benchmarks.

Produces features that better capture invariants and are less affected by style shifts.

Demonstrates improved generalization across diverse visual styles.

Abstract

We present a novel method of learning style-agnostic representation using both style transfer and adversarial learning in the reinforcement learning framework. The style, here, refers to task-irrelevant details such as the color of the background in the images, where generalizing the learned policy across environments with different styles is still a challenge. Focusing on learning style-agnostic representations, our method trains the actor with diverse image styles generated from an inherent adversarial style perturbation generator, which plays a min-max game between the actor and the generator, without demanding expert knowledge for data augmentation or additional class labels for adversarial training. We verify that our method achieves competitive or better performances than the state-of-the-art approaches on Procgen and Distracting Control Suite benchmarks, and further investigate the features extracted from our model, showing that the model better captures the invariants and is less distracted by the shifted style. The code is available at https://github.com/POSTECH-CVLab/style-agnostic-RL.