Talking-to-Build: How LLM-Assisted Interface Shapes Player Performance and Experience in Minecraft

TL;DR

This study investigates how large language model-assisted interfaces in Minecraft influence player performance and experience, demonstrating significant improvements and highlighting design considerations for AI-driven gaming environments.

Contribution

It introduces an LLM-assisted interface for Minecraft and provides empirical evidence of its positive impact on gameplay and user experience.

Findings

LLM-assisted interface enhances player performance and engagement

Task complexity affects performance and experience

Design balance is crucial for AI-driven game interfaces

Abstract

With large language models (LLMs) on the rise, in-game interactions are shifting from rigid commands to natural conversations. However, the impacts of LLMs on player performance and game experience remain underexplored. This work explores LLM's role as a co-builder during gameplay, examining its impact on task performance, usability, and player experience. Using Minecraft as a sandbox, we present an LLM-assisted interface that engages players through natural language, aiming to facilitate creativity and simplify complex gaming commands. We conducted a mixed-methods study with 30 participants, comparing LLM-assisted and command-based interfaces across simple and complex game tasks. Quantitative and qualitative analyses reveal that the LLM-assisted interface significantly improves player performance, engagement, and overall game experience. Additionally, task complexity has a notable effect on player performance and experience across both interfaces. Our findings highlight the potential of LLM-assisted interfaces to revolutionize virtual experiences, emphasizing the importance of balancing intuitiveness with predictability, transparency, and user agency in AI-driven, multimodal gaming environments.