AI-Gadget Kit: Integrating Swarm User Interfaces with LLM-driven Agents for Rich Tabletop Game Applications

TL;DR

This paper presents the AI-Gadget Kit, which integrates large language model-driven agents with Swarm User Interfaces to enable complex, autonomous interactions in tabletop games, enhancing personalization and dynamic gameplay.

Contribution

It introduces a novel framework for embedding LLM-driven agents into SUIs, including a design space, an add-on prompt method, and application scenarios for complex tabletop game interactions.

Findings

Enables complex motion planning in SUIs

Simplifies interaction design with add-on prompts

Demonstrates personalized, dynamic tabletop game interactions

Abstract

While Swarm User Interfaces (SUIs) have succeeded in enriching tangible interaction experiences, their limitations in autonomous action planning have hindered the potential for personalized and dynamic interaction generation in tabletop games. Based on the AI-Gadget Kit we developed, this paper explores how to integrate LLM-driven agents within tabletop games to enable SUIs to execute complex interaction tasks. After defining the design space of this kit, we elucidate the method for designing agents that can extend the meta-actions of SUIs to complex motion planning. Furthermore, we introduce an add-on prompt method that simplifies the design process for four interaction behaviors and four interaction relationships in tabletop games. Lastly, we present several application scenarios that illustrate the potential of AI-Gadget Kit to construct personalized interaction in SUI tabletop games. We expect to use our work as a case study to inspire research on multi-agent-driven SUI for other scenarios with complex interaction tasks.