Mitigation of Adversarial Policy Imitation via Constrained Randomization of Policy (CRoP)

TL;DR

This paper introduces CRoP, a method that randomizes policy actions within constraints to defend deep reinforcement learning policies from imitation attacks, supported by theoretical analysis and Atari experiments.

Contribution

CRoP is a novel mitigation technique that induces controlled randomness in policies to prevent unauthorized imitation, with theoretical bounds and empirical validation.

Findings

CRoP effectively reduces policy replication in Atari environments.

Theoretical bounds on adversarial budget and expected loss are established.

Experimental results show CRoP's efficacy against imitation attacks.

Abstract

Deep reinforcement learning (DRL) policies are vulnerable to unauthorized replication attacks, where an adversary exploits imitation learning to reproduce target policies from observed behavior. In this paper, we propose Constrained Randomization of Policy (CRoP) as a mitigation technique against such attacks. CRoP induces the execution of sub-optimal actions at random under performance loss constraints. We present a parametric analysis of CRoP, address the optimality of CRoP, and establish theoretical bounds on the adversarial budget and the expectation of loss. Furthermore, we report the experimental evaluation of CRoP in Atari environments under adversarial imitation, which demonstrate the efficacy and feasibility of our proposed method against policy replication attacks.