Designing and Evaluating Dialogue LLMs for Co-Creative Improvised Theatre

TL;DR

This study evaluates Large Language Models in a live co-creative theatre setting, highlighting technical challenges, audience perceptions, and the potential of AI as a tool for artistic improvisation in a real-world performance.

Contribution

It presents a novel case study of deploying LLMs in a professional theatre environment, providing insights into human-AI interaction and technical constraints during live improvisational performances.

Findings

Audience interest in AI-driven entertainment increased during the show.

Performers showed enthusiasm and varied satisfaction with AI collaboration.

Technical challenges in generating contextually relevant responses were identified.

Abstract

Social robotics researchers are increasingly interested in multi-party trained conversational agents. With a growing demand for real-world evaluations, our study presents Large Language Models (LLMs) deployed in a month-long live show at the Edinburgh Festival Fringe. This case study investigates human improvisers co-creating with conversational agents in a professional theatre setting. We explore the technical capabilities and constraints of on-the-spot multi-party dialogue, providing comprehensive insights from both audience and performer experiences with AI on stage. Our human-in-the-loop methodology underlines the challenges of these LLMs in generating context-relevant responses, stressing the user interface's crucial role. Audience feedback indicates an evolving interest for AI-driven live entertainment, direct human-AI interaction, and a diverse range of expectations about AI's conversational competence and utility as a creativity support tool. Human performers express immense enthusiasm, varied satisfaction, and the evolving public opinion highlights mixed emotions about AI's role in arts.