HAMLET: A Hierarchical and Adaptive Multi-Agent Framework for Live Embodied Theatrics

TL;DR

HAMLET is a hierarchical multi-agent framework leveraging LLMs for immersive, real-time embodied theatrical performances with adaptive decision-making and automated quality evaluation.

Contribution

It introduces a novel multi-agent system for live drama creation and performance, integrating embodied interactions and a new evaluation method.

Findings

HAMLET generates coherent and expressive theatrical narratives.

Actors adaptively interact with scene props and each other.

HAMLETJudge effectively evaluates live embodied theatrics.

Abstract

Creating an immersive and interactive theatrical experience is a long-term goal in the field of interactive narrative. The emergence of large language models (LLMs) provides a new path to achieve this goal. However, existing drama generation methods often produce LLMs that lack initiative and cannot interact with the physical scene, while typically requiring detailed input that diminishes the immersion of live performance. To address these challenges, we propose HAMLET, a hierarchical adaptive multi-agent framework focused on drama creation and real-time online performance. Given a simple topic, the framework initially generates a narrative blueprint to guide the subsequent improvisational performance. During online performance, each actor is equipped with an adaptive reasoning module that enables decision-making based on their personas, memories, goals during complex group chat scenarios. Beyond dialogue, actor agents engage in embodied interactions by changing the state of scene props through actions such as opening a letter or picking up a weapon, which are broadcast to update the global environmental context. To objectively assess the quality of live embodied theatrics, we establish a comprehensive evaluation method and introduce HAMLETJudge, a specialized critic model for automated evaluation. Experimental results demonstrate that HAMLET excels in creating expressive, coherent, and physically interactive theatrical experiences in an autonomous manner.