Different but Equal: Comparing User Collaboration with Digital Personal Assistants vs. Teams of Expert Agents

TL;DR

This study compares user collaboration with personal assistants versus expert chatbot teams, finding both approaches equally effective in task accomplishment and collaboration costs, despite different user experiences.

Contribution

It provides empirical evidence that both single-agent and multi-agent chatbot systems are similarly effective for user collaboration in financial advice tasks.

Findings

Users achieved goals equally with both systems.

Users predicted agent behavior better in team settings.

Collaboration costs were similar across approaches.

Abstract

This work compares user collaboration with conversational personal assistants vs. teams of expert chatbots. Two studies were performed to investigate whether each approach affects accomplishment of tasks and collaboration costs. Participants interacted with two equivalent financial advice chatbot systems, one composed of a single conversational adviser and the other based on a team of four experts chatbots. Results indicated that users had different forms of experiences but were equally able to achieve their goals. Contrary to the expected, there were evidences that in the teamwork situation that users were more able to predict agent behavior better and did not have an overhead to maintain common ground, indicating similar collaboration costs. The results point towards the feasibility of either of the two approaches for user collaboration with conversational agents.