Contrastive explanations of BDI agents

TL;DR

This paper extends BDI agent explanations to contrastive questions, demonstrating that contrastive answers are shorter and potentially more trusted, with mixed results on the overall benefit of explanations.

Contribution

It introduces a method for BDI agents to answer contrastive questions, enhancing explanation quality and supporting trust and transparency.

Findings

Contrastive answers are significantly shorter.

Some evidence suggests contrastive answers increase trust.

Providing explanations does not always improve understanding.

Abstract

The ability of autonomous systems to provide explanations is important for supporting transparency and aiding the development of (appropriate) trust. Prior work has defined a mechanism for Belief-Desire-Intention (BDI) agents to be able to answer questions of the form ``why did you do action $X$?''. However, we know that we ask \emph{contrastive} questions (``why did you do $X$ \emph{instead of} $F$?''). We therefore extend previous work to be able to answer such questions. A computational evaluation shows that using contrastive questions yields a significant reduction in explanation length. A human subject evaluation was conducted to assess whether such contrastive answers are preferred, and how well they support trust development and transparency. We found some evidence for contrastive answers being preferred, and some evidence that they led to higher trust, perceived understanding, and confidence in the system's correctness. We also evaluated the benefit of providing explanations at all. Surprisingly, there was not a clear benefit, and in some situations we found evidence that providing a (full) explanation was worse than not providing any explanation.