I Probe, Therefore I Am: Designing a Virtual Journalist with Human Emotions

TL;DR

This paper presents a Virtual Human Journalist that uses multimodal communication to interact naturally with users, demonstrating technological feasibility while exploring the impact of naturalness and personality cues on user engagement.

Contribution

It introduces a humanoid conversational agent employing novel NLP, computer vision, and speech synthesis techniques, and discusses the effects of naturalness and personality cues on interaction quality.

Findings

System is technically feasible based on evaluation results.

Naturalness is not always necessary for effective engagement.

Altered personality cues can enhance user interaction.

Abstract

By utilizing different communication channels, such as verbal language, gestures or facial expressions, virtually embodied interactive humans hold a unique potential to bridge the gap between human-computer interaction and actual interhuman communication. The use of virtual humans is consequently becoming increasingly popular in a wide range of areas where such a natural communication might be beneficial, including entertainment, education, mental health research and beyond. Behind this development lies a series of technological advances in a multitude of disciplines, most notably natural language processing, computer vision, and speech synthesis. In this paper we discuss a Virtual Human Journalist, a project employing a number of novel solutions from these disciplines with the goal to demonstrate their viability by producing a humanoid conversational agent capable of naturally eliciting and reacting to information from a human user. A set of qualitative and quantitative evaluation sessions demonstrated the technical feasibility of the system whilst uncovering a number of deficits in its capacity to engage users in a way that would be perceived as natural and emotionally engaging. We argue that naturalness should not always be seen as a desirable goal and suggest that deliberately suppressing the naturalness of virtual human interactions, such as by altering its personality cues, might in some cases yield more desirable results.