Detecting Agreement in Multi-party Conversational AI

TL;DR

This paper presents a multi-party conversational system for Socially Assistive Robots that detects user agreement or disagreement during a trivia game, addressing challenges like speaker and addressee recognition.

Contribution

It introduces a system capable of detecting agreement in multi-party conversations, with open-source code and evaluation results included.

Findings

Effective agreement detection in multi-party settings

Open-source system and annotated transcripts available

Positive user assessment outcomes

Abstract

Today, conversational systems are expected to handle conversations in multi-party settings, especially within Socially Assistive Robots (SARs). However, practical usability remains difficult as there are additional challenges to overcome, such as speaker recognition, addressee recognition, and complex turn-taking. In this paper, we present our work on a multi-party conversational system, which invites two users to play a trivia quiz game. The system detects users' agreement or disagreement on a final answer and responds accordingly. Our evaluation includes both performance and user assessment results, with a focus on detecting user agreement. Our annotated transcripts and the code for the proposed system have been released open-source on GitHub.