Adversarial Inception Backdoor Attacks against Reinforcement Learning

TL;DR

This paper introduces a novel backdoor attack method called inception attacks against Deep Reinforcement Learning, achieving high success rates under strict reward constraints by manipulating training data to induce adversarial behaviors.

Contribution

The work presents the first inception attack method that maintains high attack success under reward constraints, formally defines the attack, and demonstrates its effectiveness across multiple environments.

Findings

100% attack success rate in tested environments

Minimal impact on agent's original task performance

Effective under strict reward constraints

Abstract

Recent works have demonstrated the vulnerability of Deep Reinforcement Learning (DRL) algorithms against training-time, backdoor poisoning attacks. The objectives of these attacks are twofold: induce pre-determined, adversarial behavior in the agent upon observing a fixed trigger during deployment while allowing the agent to solve its intended task during training. Prior attacks assume arbitrary control over the agent's rewards, inducing values far outside the environment's natural constraints. This results in brittle attacks that fail once the proper reward constraints are enforced. Thus, in this work we propose a new class of backdoor attacks against DRL which are the first to achieve state of the art performance under strict reward constraints. These "inception" attacks manipulate the agent's training data -- inserting the trigger into prior observations and replacing high return actions with those of the targeted adversarial behavior. We formally define these attacks and prove they achieve both adversarial objectives against arbitrary Markov Decision Processes (MDP). Using this framework we devise an online inception attack which achieves an 100\% attack success rate on multiple environments under constrained rewards while minimally impacting the agent's task performance.