New Challenges in Reinforcement Learning: A Survey of Security and Privacy

TL;DR

This survey reviews security and privacy challenges in reinforcement learning, analyzing threats and solutions from the perspective of Markov Decision Processes, and discusses future research directions.

Contribution

It provides a systematic, comprehensive review of security and privacy issues in reinforcement learning from an MDP perspective, filling a gap in existing literature.

Findings

Identifies key security and privacy vulnerabilities in RL systems.

Summarizes state-of-the-art solutions and methodologies.

Highlights future research directions in the field.

Abstract

Reinforcement learning (RL) is one of the most important branches of AI. Due to its capacity for self-adaption and decision-making in dynamic environments, reinforcement learning has been widely applied in multiple areas, such as healthcare, data markets, autonomous driving, and robotics. However, some of these applications and systems have been shown to be vulnerable to security or privacy attacks, resulting in unreliable or unstable services. A large number of studies have focused on these security and privacy problems in reinforcement learning. However, few surveys have provided a systematic review and comparison of existing problems and state-of-the-art solutions to keep up with the pace of emerging threats. Accordingly, we herein present such a comprehensive review to explain and summarize the challenges associated with security and privacy in reinforcement learning from a new perspective, namely that of the Markov Decision Process (MDP). In this survey, we first introduce the key concepts related to this area. Next, we cover the security and privacy issues linked to the state, action, environment, and reward function of the MDP process, respectively. We further highlight the special characteristics of security and privacy methodologies related to reinforcement learning. Finally, we discuss the possible future research directions within this area.