Unified State Representation Learning under Data Augmentation

TL;DR

USRA is a novel reinforcement learning framework that uses data augmentation to learn a unified state representation, significantly enhancing generalization and domain adaptation in visual tasks.

Contribution

The paper introduces USRA, a data augmentation-based representation learning method that improves zero-shot transfer and generalization in RL environments.

Findings

USRA achieves 14.3% better domain adaptation performance.

USRA demonstrates higher sample efficiency.

Effective generalization on DeepMind Control Benchmark.

Abstract

The capacity for rapid domain adaptation is important to increasing the applicability of reinforcement learning (RL) to real world problems. Generalization of RL agents is critical to success in the real world, yet zero-shot policy transfer is a challenging problem since even minor visual changes could make the trained agent completely fail in the new task. We propose USRA: Unified State Representation Learning under Data Augmentation, a representation learning framework that learns a latent unified state representation by performing data augmentations on its observations to improve its ability to generalize to unseen target domains. We showcase the success of our approach on the DeepMind Control Generalization Benchmark for the Walker environment and find that USRA achieves higher sample efficiency and 14.3% better domain adaptation performance compared to the best baseline results.