Domain Adversarial Reinforcement Learning

TL;DR

This paper introduces a domain adversarial reinforcement learning method that enhances the generalization ability of agents to unseen visual domains by enforcing invariant representations through adversarial training.

Contribution

It proposes a novel domain adversarial training approach for reinforcement learning that improves zero-shot generalization to unseen visual environments.

Findings

Significant improvement in generalization to unseen domains.

Effective invariance of learned representations across visual variations.

Applicable to various visual domain shifts in RL environments.

Abstract

We consider the problem of generalization in reinforcement learning where visual aspects of the observations might differ, e.g. when there are different backgrounds or change in contrast, brightness, etc. We assume that our agent has access to only a few of the MDPs from the MDP distribution during training. The performance of the agent is then reported on new unknown test domains drawn from the distribution (e.g. unseen backgrounds). For this "zero-shot RL" task, we enforce invariance of the learned representations to visual domains via a domain adversarial optimization process. We empirically show that this approach allows achieving a significant generalization improvement to new unseen domains.