Development of a Trust-Aware User Simulator for Statistical Proactive Dialog Modeling in Human-AI Teams

TL;DR

This paper introduces a trust-aware user simulator designed for training and testing proactive dialog policies in Human-AI teams, focusing on modeling user trust and behavior to improve collaboration.

Contribution

It develops a corpus-based user simulator that incorporates trust and personal traits, comparing two approaches and demonstrating the effectiveness of a task-step-based method.

Findings

Task-step-based approach outperforms other methods

Simulator effectively models user trust and behavior

Promising for evaluating proactive dialog strategies

Abstract

The concept of a Human-AI team has gained increasing attention in recent years. For effective collaboration between humans and AI teammates, proactivity is crucial for close coordination and effective communication. However, the design of adequate proactivity for AI-based systems to support humans is still an open question and a challenging topic. In this paper, we present the development of a corpus-based user simulator for training and testing proactive dialog policies. The simulator incorporates informed knowledge about proactive dialog and its effect on user trust and simulates user behavior and personal information, including socio-demographic features and personality traits. Two different simulation approaches were compared, and a task-step-based approach yielded better overall results due to enhanced modeling of sequential dependencies. This research presents a promising avenue for exploring and evaluating appropriate proactive strategies in a dialog game setting for improving Human-AI teams.