SAPIEN: Affective Virtual Agents Powered by Large Language Models

TL;DR

SAPIEN is a versatile platform that creates emotionally expressive virtual agents powered by large language models, supporting multilingual open conversations, customization, and post-interaction analysis for diverse applications.

Contribution

This paper introduces SAPIEN, a novel platform integrating large language models with emotional expression and customization for virtual agents across multiple domains.

Findings

Supports 13 languages for open domain conversations

Enables emotion display through facial expressions and voice

Provides post-conversation communication skill analysis

Abstract

In this demo paper, we introduce SAPIEN, a platform for high-fidelity virtual agents driven by large language models that can hold open domain conversations with users in 13 different languages, and display emotions through facial expressions and voice. The platform allows users to customize their virtual agent's personality, background, and conversation premise, thus providing a rich, immersive interaction experience. Furthermore, after the virtual meeting, the user can choose to get the conversation analyzed and receive actionable feedback on their communication skills. This paper illustrates an overview of the platform and discusses the various application domains of this technology, ranging from entertainment to mental health, communication training, language learning, education, healthcare, and beyond. Additionally, we consider the ethical implications of such realistic virtual agent representations and the potential challenges in ensuring responsible use.