Explanation of Reinforcement Learning Model in Dynamic Multi-Agent System

TL;DR

This paper introduces a novel approach for generating verbal explanations in deep reinforcement learning systems, combining rule-based and learning models to enhance interpretability and user satisfaction.

Contribution

It presents a hybrid explanation generation framework that improves flexibility and generalizability over static rule-based methods in DRL systems.

Findings

Verbal explanations increase user satisfaction and trust.

Learning models outperform static rule-based explanations in flexibility.

Multiple model variants demonstrate the impact of design components on explanation quality.

Abstract

Recently, there has been increasing interest in transparency and interpretability in Deep Reinforcement Learning (DRL) systems. Verbal explanations, as the most natural way of communication in our daily life, deserve more attention, since they allow users to gain a better understanding of the system which ultimately could lead to a high level of trust and smooth collaboration. This paper reports a novel work in generating verbal explanations for DRL behaviors agent. A rule-based model is designed to construct explanations using a series of rules which are predefined with prior knowledge. A learning model is then proposed to expand the implicit logic of generating verbal explanation to general situations by employing rule-based explanations as training data. The learning model is shown to have better flexibility and generalizability than the static rule-based model. The performance of both models is evaluated quantitatively through objective metrics. The results show that verbal explanation generated by both models improve subjective satisfaction of users towards the interpretability of DRL systems. Additionally, seven variants of the learning model are designed to illustrate the contribution of input channels, attention mechanism, and proposed encoder in improving the quality of verbal explanation.