A Platform for Interactive AI Character Experiences

TL;DR

This paper introduces a comprehensive platform that integrates multiple AI technologies to create interactive, believable digital characters capable of engaging in story-driven conversations, exemplified by a digital Einstein.

Contribution

The authors present a unified, adaptable platform that combines various AI components to facilitate the development of interactive, story-driven digital characters, addressing a complex integration challenge.

Findings

Successfully created Digital Einstein for engaging conversations

Platform generalizes to any story-driven character

Enables immersive, lifelike character interactions

Abstract

From movie characters to modern science fiction - bringing characters into interactive, story-driven conversations has captured imaginations across generations. Achieving this vision is highly challenging and requires much more than just language modeling. It involves numerous complex AI challenges, such as conversational AI, maintaining character integrity, managing personality and emotions, handling knowledge and memory, synthesizing voice, generating animations, enabling real-world interactions, and integration with physical environments. Recent advancements in the development of foundation models, prompt engineering, and fine-tuning for downstream tasks have enabled researchers to address these individual challenges. However, combining these technologies for interactive characters remains an open problem. We present a system and platform for conveniently designing believable digital characters, enabling a conversational and story-driven experience while providing solutions to all of the technical challenges. As a proof-of-concept, we introduce Digital Einstein, which allows users to engage in conversations with a digital representation of Albert Einstein about his life, research, and persona. While Digital Einstein exemplifies our methods for a specific character, our system is flexible and generalizes to any story-driven or conversational character. By unifying these diverse AI components into a single, easy-to-adapt platform, our work paves the way for immersive character experiences, turning the dream of lifelike, story-based interactions into a reality.