Investigating Perceptions of Social Intelligence in Simulated Human-Chatbot Interactions

TL;DR

This study explores whether humans can perceive social intelligence traits like authenticity, clarity, and empathy in chatbot interactions and how these perceptions affect dialogue quality, revealing challenges in perceiving social cues in text-based AI.

Contribution

It investigates the perceivability of social intelligence in chatbots and its impact on interaction quality, highlighting difficulties and potential effects of anthropomorphic behavior.

Findings

Humans struggle to perceive social intelligence in chatbot text.

Anthropomorphic behavior enhances naturalness but can be perceived as frightening.

Perception of social cues influences dialogue experience.

Abstract

With the ongoing penetration of conversational user interfaces, a better understanding of social and emotional characteristic inherent to dialogue is required. Chatbots in particular face the challenge of conveying human-like behaviour while being restricted to one channel of interaction, i.e., text. The goal of the presented work is thus to investigate whether characteristics of social intelligence embedded in human-chatbot interactions are perceivable by human interlocutors and if yes, whether such influences the experienced interaction quality. Focusing on the social intelligence dimensions Authenticity, Clarity and Empathy, we first used a questionnaire survey evaluating the level of perception in text utterances, and then conducted a Wizard of Oz study to investigate the effects of these utterances in a more interactive setting. Results show that people have great difficulties perceiving elements of social intelligence in text. While on the one hand they find anthropomorphic behaviour pleasant and positive for the naturalness of a dialogue, they may also perceive it as frightening and unsuitable when expressed by an artificial agent in the wrong way or at the wrong time.