Explainable Human-AI Interaction: A Planning Perspective

TL;DR

This paper discusses how explainable AI systems can effectively communicate with humans by modeling human mental states, enhancing cooperation and understanding in human-AI interactions.

Contribution

It introduces a planning perspective on explainable AI, emphasizing the importance of modeling human mental states for better communication and cooperation.

Findings

AI agents can conform to or change human expectations through explanations.

Modeling human mental states improves AI-human collaboration.

The approach applies to cooperative, obfuscation, and deception scenarios.

Abstract

From its inception, AI has had a rather ambivalent relationship with humans -- swinging between their augmentation and replacement. Now, as AI technologies enter our everyday lives at an ever increasing pace, there is a greater need for AI systems to work synergistically with humans. One critical requirement for such synergistic human-AI interaction is that the AI systems be explainable to the humans in the loop. To do this effectively, AI agents need to go beyond planning with their own models of the world, and take into account the mental model of the human in the loop. Drawing from several years of research in our lab, we will discuss how the AI agent can use these mental models to either conform to human expectations, or change those expectations through explanatory communication. While the main focus of the book is on cooperative scenarios, we will point out how the same mental models can be used for obfuscation and deception. Although the book is primarily driven by our own research in these areas, in every chapter, we will provide ample connections to relevant research from other groups.