# Generation of Policy-Level Explanations for Reinforcement Learning

## TL;DR

This paper introduces Abstracted Policy Graphs, a novel method for explaining entire reinforcement learning policies at a high level, improving interpretability and verification of sequential decision-making systems.

## Contribution

We propose a new approach to generate policy-level explanations using Abstracted Policy Graphs, compatible with various RL methods and capable of summarizing complex policies efficiently.

## Key findings

- Method scales well with domain size
- Graphs reliably capture domain relationships
- Compatible with multiple RL algorithms

## Abstract

Though reinforcement learning has greatly benefited from the incorporation of neural networks, the inability to verify the correctness of such systems limits their use. Current work in explainable deep learning focuses on explaining only a single decision in terms of input features, making it unsuitable for explaining a sequence of decisions. To address this need, we introduce Abstracted Policy Graphs, which are Markov chains of abstract states. This representation concisely summarizes a policy so that individual decisions can be explained in the context of expected future transitions. Additionally, we propose a method to generate these Abstracted Policy Graphs for deterministic policies given a learned value function and a set of observed transitions, potentially off-policy transitions used during training. Since no restrictions are placed on how the value function is generated, our method is compatible with many existing reinforcement learning methods. We prove that the worst-case time complexity of our method is quadratic in the number of features and linear in the number of provided transitions, $O(|F|^2 |tr\_samples|)$. By applying our method to a family of domains, we show that our method scales well in practice and produces Abstracted Policy Graphs which reliably capture relationships within these domains.