Complementary reinforcement learning towards explainable agents

TL;DR

This paper proposes a method to create a secondary, interpretable agent from a neural network-based reinforcement learning agent, enhancing transparency without significantly sacrificing performance.

Contribution

It introduces a novel approach to derive a comprehensible agent from a neural RL agent, addressing the explainability challenge in deep reinforcement learning.

Findings

Secondary agent's decision-making based on simple rules

Empirical evaluation supports the feasibility of transparent RL agents

Potential for deploying explainable RL in real-world applications

Abstract

Reinforcement learning (RL) algorithms allow agents to learn skills and strategies to perform complex tasks without detailed instructions or expensive labelled training examples. That is, RL agents can learn, as we learn. Given the importance of learning in our intelligence, RL has been thought to be one of key components to general artificial intelligence, and recent breakthroughs in deep reinforcement learning suggest that neural networks (NN) are natural platforms for RL agents. However, despite the efficiency and versatility of NN-based RL agents, their decision-making remains incomprehensible, reducing their utilities. To deploy RL into a wider range of applications, it is imperative to develop explainable NN-based RL agents. Here, we propose a method to derive a secondary comprehensible agent from a NN-based RL agent, whose decision-makings are based on simple rules. Our empirical evaluation of this secondary agent's performance supports the possibility of building a comprehensible and transparent agent using a NN-based RL agent.