ReRoGCRL: Representation-based Robustness in Goal-Conditioned Reinforcement Learning

TL;DR

This paper introduces a novel attack and defense framework for Goal-Conditioned Reinforcement Learning, enhancing robustness against adversarial perturbations through semi-contrastive attacks and regularization techniques, validated across multiple algorithms.

Contribution

It proposes the Semi-Contrastive Representation attack and Adversarial Representation Tactics, novel methods to evaluate and improve GCRL robustness against adversarial attacks.

Findings

The attack effectively compromises GCRL policies during deployment.

The defense significantly improves robustness against various perturbations.

Experimental results outperform existing methods in robustness metrics.

Abstract

While Goal-Conditioned Reinforcement Learning (GCRL) has gained attention, its algorithmic robustness against adversarial perturbations remains unexplored. The attacks and robust representation training methods that are designed for traditional RL become less effective when applied to GCRL. To address this challenge, we first propose the Semi-Contrastive Representation attack, a novel approach inspired by the adversarial contrastive attack. Unlike existing attacks in RL, it only necessitates information from the policy function and can be seamlessly implemented during deployment. Then, to mitigate the vulnerability of existing GCRL algorithms, we introduce Adversarial Representation Tactics, which combines Semi-Contrastive Adversarial Augmentation with Sensitivity-Aware Regularizer to improve the adversarial robustness of the underlying RL agent against various types of perturbations. Extensive experiments validate the superior performance of our attack and defence methods across multiple state-of-the-art GCRL algorithms. Our tool ReRoGCRL is available at https://github.com/TrustAI/ReRoGCRL.