Review of Metrics to Measure the Stability, Robustness and Resilience of Reinforcement Learning

TL;DR

This paper provides a comprehensive review of metrics used to evaluate the stability, robustness, and resilience of reinforcement learning methods, highlighting current approaches and offering guidance for metric selection.

Contribution

It is the first detailed review focusing specifically on metrics for stability, robustness, and resilience in reinforcement learning, including a decision tree for metric selection.

Findings

Classified various approaches to measure robustness, stability, and resilience.

Identified key actions or events targeted by these metrics.

Provided a decision tree to aid in selecting appropriate metrics.

Abstract

Reinforcement learning has received significant interest in recent years, due primarily to the successes of deep reinforcement learning at solving many challenging tasks such as playing Chess, Go and online computer games. However, with the increasing focus on reinforcement learning, applications outside of gaming and simulated environments require understanding the robustness, stability, and resilience of reinforcement learning methods. To this end, we conducted a comprehensive literature review to characterize the available literature on these three behaviors as they pertain to reinforcement learning. We classify the quantitative and theoretical approaches used to indicate or measure robustness, stability, and resilience behaviors. In addition, we identified the action or event to which the quantitative approaches were attempting to be stable, robust, or resilient. Finally, we provide a decision tree useful for selecting metrics to quantify the behaviors. We believe that this is the first comprehensive review of stability, robustness and resilience specifically geared towards reinforcement learning.