Membership Inference Attacks Against Temporally Correlated Data in Deep Reinforcement Learning

TL;DR

This paper introduces a novel adversarial attack framework targeting deep reinforcement learning models, demonstrating high success rates in inferring training data, especially considering temporal correlations, thus highlighting privacy vulnerabilities.

Contribution

The paper presents the first tailored membership inference attack framework for deep reinforcement learning, analyzing the effects of temporal correlation and learning state on privacy risks.

Findings

Achieved over 84% accuracy in individual attacks and 97% in collective attacks.

Temporal correlation increases vulnerability to membership inference.

Learning state significantly affects privacy breach levels.

Abstract

While significant research advances have been made in the field of deep reinforcement learning, there have been no concrete adversarial attack strategies in literature tailored for studying the vulnerability of deep reinforcement learning algorithms to membership inference attacks. In such attacking systems, the adversary targets the set of collected input data on which the deep reinforcement learning algorithm has been trained. To address this gap, we propose an adversarial attack framework designed for testing the vulnerability of a state-of-the-art deep reinforcement learning algorithm to a membership inference attack. In particular, we design a series of experiments to investigate the impact of temporal correlation, which naturally exists in reinforcement learning training data, on the probability of information leakage. Moreover, we compare the performance of \emph{collective} and \emph{individual} membership attacks against the deep reinforcement learning algorithm. Experimental results show that the proposed adversarial attack framework is surprisingly effective at inferring data with an accuracy exceeding $84\%$ in individual and $97\%$ in collective modes in three different continuous control Mujoco tasks, which raises serious privacy concerns in this regard. Finally, we show that the learning state of the reinforcement learning algorithm influences the level of privacy breaches significantly.