Mitigating Adversarial Perturbations for Deep Reinforcement Learning via Vector Quantization

TL;DR

This paper introduces a vector quantization-based input transformation method to improve the robustness of deep reinforcement learning agents against adversarial perturbations, offering an efficient and integrable defense mechanism.

Contribution

The paper proposes a novel input transformation using vector quantization to defend RL agents from adversarial attacks, complementing existing adversarial training methods.

Findings

VQ-based transformation reduces attack success rate.

Method is computationally efficient and easy to integrate.

Significant robustness improvement demonstrated across multiple environments.

Abstract

Recent studies reveal that well-performing reinforcement learning (RL) agents in training often lack resilience against adversarial perturbations during deployment. This highlights the importance of building a robust agent before deploying it in the real world. Most prior works focus on developing robust training-based procedures to tackle this problem, including enhancing the robustness of the deep neural network component itself or adversarially training the agent on strong attacks. In this work, we instead study an input transformation-based defense for RL. Specifically, we propose using a variant of vector quantization (VQ) as a transformation for input observations, which is then used to reduce the space of adversarial attacks during testing, resulting in the transformed observations being less affected by attacks. Our method is computationally efficient and seamlessly integrates with adversarial training, further enhancing the robustness of RL agents against adversarial attacks. Through extensive experiments in multiple environments, we demonstrate that using VQ as the input transformation effectively defends against adversarial attacks on the agent's observations.