(When) Are Contrastive Explanations of Reinforcement Learning Helpful?

TL;DR

This paper investigates the usefulness of contrastive explanations in reinforcement learning, finding that complete explanations often outperform contrastive ones unless they are larger, indicating the need for further research on their effectiveness.

Contribution

The study provides empirical insights into when contrastive explanations are beneficial in RL, highlighting their limitations compared to complete explanations.

Findings

Complete explanations are more effective when same size or smaller than contrastive explanations.

Contrastive explanations are not always sufficient for effective RL policy explanations.

Further research is needed to understand the optimal use of contrastive explanations in RL.

Abstract

Global explanations of a reinforcement learning (RL) agent's expected behavior can make it safer to deploy. However, such explanations are often difficult to understand because of the complicated nature of many RL policies. Effective human explanations are often contrastive, referencing a known contrast (policy) to reduce redundancy. At the same time, these explanations also require the additional effort of referencing that contrast when evaluating an explanation. We conduct a user study to understand whether and when contrastive explanations might be preferable to complete explanations that do not require referencing a contrast. We find that complete explanations are generally more effective when they are the same size or smaller than a contrastive explanation of the same policy, and no worse when they are larger. This suggests that contrastive explanations are not sufficient to solve the problem of effectively explaining reinforcement learning policies, and require additional careful study for use in this context.