Approximating Human Models During Argumentation-based Dialogues

TL;DR

This paper introduces a framework for AI agents to learn and update probabilistic human mental models during argumentation dialogues, enhancing explainability and trust in human-AI interactions.

Contribution

It presents a novel probabilistic model updating method using trust and certainty signals, incorporating prospect theory and Bayesian inference in argumentation-based dialogues.

Findings

Effective modeling of human belief dynamics

Improved alignment of AI and human mental models

Enhanced trust and collaboration in human-AI interactions

Abstract

Explainable AI Planning (XAIP) aims to develop AI agents that can effectively explain their decisions and actions to human users, fostering trust and facilitating human-AI collaboration. A key challenge in XAIP is model reconciliation, which seeks to align the mental models of AI agents and humans. While existing approaches often assume a known and deterministic human model, this simplification may not capture the complexities and uncertainties of real-world interactions. In this paper, we propose a novel framework that enables AI agents to learn and update a probabilistic human model through argumentation-based dialogues. Our approach incorporates trust-based and certainty-based update mechanisms, allowing the agent to refine its understanding of the human's mental state based on the human's expressed trust in the agent's arguments and certainty in their own arguments. We employ a probability weighting function inspired by prospect theory to capture the relationship between trust and perceived probability, and use a Bayesian approach to update the agent's probability distribution over possible human models. We conduct a human-subject study to empirically evaluate the effectiveness of our approach in an argumentation scenario, demonstrating its ability to capture the dynamics of human belief formation and adaptation.