AVIN-Chat: An Audio-Visual Interactive Chatbot System with Emotional State Tuning

TL;DR

AVIN-Chat is an innovative audio-visual chatbot system that enables real-time face-to-face interactions with 3D avatars, emotionally responsive speech, and expressions, significantly enhancing user immersion and emotional connection.

Contribution

This work introduces AVIN-Chat, a novel system combining audio-visual communication and emotional expression in chatbots, surpassing traditional text or speech-only interfaces.

Findings

Users reported higher immersion levels with AVIN-Chat.

The system successfully integrates emotional speaking and expressions.

User tests confirmed improved engagement over previous chatbots.

Abstract

This work presents an audio-visual interactive chatbot (AVIN-Chat) system that allows users to have face-to-face conversations with 3D avatars in real-time. Compared to the previous chatbot services, which provide text-only or speech-only communications, the proposed AVIN-Chat can offer audio-visual communications providing users with a superior experience quality. In addition, the proposed AVIN-Chat emotionally speaks and expresses according to the user's emotional state. Thus, it enables users to establish a strong bond with the chatbot system, increasing the user's immersion. Through user subjective tests, it is demonstrated that the proposed system provides users with a higher sense of immersion than previous chatbot systems. The demonstration video is available at https://www.youtube.com/watch?v=Z74uIV9k7_k.