Intelligent Conversational Android ERICA Applied to Attentive Listening and Job Interview

TL;DR

This paper presents ERICA, an advanced humanoid robot capable of engaging in human-like, long, and deep conversations through attentive listening and job interview tasks, utilizing robust turn-taking and backchannel prediction models.

Contribution

The development of ERICA with novel turn-taking, backchannel prediction, and open-domain conversational capabilities for social interaction tasks.

Findings

Successful long conversations with seniors without breakdowns

Effective open-domain attentive listening system

Promising results in robot-led job interview scenarios

Abstract

Following the success of spoken dialogue systems (SDS) in smartphone assistants and smart speakers, a number of communicative robots are developed and commercialized. Compared with the conventional SDSs designed as a human-machine interface, interaction with robots is expected to be in a closer manner to talking to a human because of the anthropomorphism and physical presence. The goal or task of dialogue may not be information retrieval, but the conversation itself. In order to realize human-level "long and deep" conversation, we have developed an intelligent conversational android ERICA. We set up several social interaction tasks for ERICA, including attentive listening, job interview, and speed dating. To allow for spontaneous, incremental multiple utterances, a robust turn-taking model is implemented based on TRP (transition-relevance place) prediction, and a variety of backchannels are generated based on time frame-wise prediction instead of IPU-based prediction. We have realized an open-domain attentive listening system with partial repeats and elaborating questions on focus words as well as assessment responses. It has been evaluated with 40 senior people, engaged in conversation of 5-7 minutes without a conversation breakdown. It was also compared against the WOZ setting. We have also realized a job interview system with a set of base questions followed by dynamic generation of elaborating questions. It has also been evaluated with student subjects, showing promising results.