UX Research on Conversational Human-AI Interaction: A Literature Review of the ACM Digital Library

TL;DR

This literature review examines UX research on polyadic human-AI interactions, highlighting key design considerations like social boundaries and proposing evaluation metrics to guide future development of ethical and usable conversational agents.

Contribution

It synthesizes scattered research on polyadic CAs, introduces evaluation measurements, and emphasizes the importance of social boundaries for ethical design.

Findings

Designing with social boundaries is crucial for ethics.

Evaluation measurements for polyadic CAs are proposed.

Future work should focus on usability testing and trust-building.

Abstract

Early conversational agents (CAs) focused on dyadic human-AI interaction between humans and the CAs, followed by the increasing popularity of polyadic human-AI interaction, in which CAs are designed to mediate human-human interactions. CAs for polyadic interactions are unique because they encompass hybrid social interactions, i.e., human-CA, human-to-human, and human-to-group behaviors. However, research on polyadic CAs is scattered across different fields, making it challenging to identify, compare, and accumulate existing knowledge. To promote the future design of CA systems, we conducted a literature review of ACM publications and identified a set of works that conducted UX (user experience) research. We qualitatively synthesized the effects of polyadic CAs into four aspects of human-human interactions, i.e., communication, engagement, connection, and relationship maintenance. Through a mixed-method analysis of the selected polyadic and dyadic CA studies, we developed a suite of evaluation measurements on the effects. Our findings show that designing with social boundaries, such as privacy, disclosure, and identification, is crucial for ethical polyadic CAs. Future research should also advance usability testing methods and trust-building guidelines for conversational AI.