Malinowski in the Age of AI: Can large language models create a text game based on an anthropological classic?

TL;DR

This study investigates whether large language models can autonomously create anthropologically themed text-based games, assessing their effectiveness and limitations through iterative design and expert playtesting.

Contribution

It demonstrates the potential and challenges of using LLMs to generate culturally and academically meaningful text games based on anthropological classics.

Findings

LLMs can generate initial game prototypes based on anthropological texts.

Models face difficulties in conveying deep thematic understanding and accurate biographical details.

Playtesting reveals issues with misinformation, monotony, and limited depth in responses.

Abstract

Recent advancements in Large Language Models (LLMs) like ChatGPT and GPT-4 have shown remarkable abilities in a wide range of tasks such as summarizing texts and assisting in coding. Scientific research has demonstrated that these models can also play text-adventure games. This study aims to explore whether LLMs can autonomously create text-based games based on anthropological classics, evaluating also their effectiveness in communicating knowledge. To achieve this, the study engaged anthropologists in discussions to gather their expectations and design inputs for an anthropologically themed game. Through iterative processes following the established HCI principle of 'design thinking', the prompts and the conceptual framework for crafting these games were refined. Leveraging GPT3.5, the study created three prototypes of games centered around the seminal anthropological work of the social anthropologist's Bronislaw Malinowski's "Argonauts of the Western Pacific" (1922). Subsequently, evaluations were conducted by inviting senior anthropologists to playtest these games, and based on their inputs, the game designs were refined. The tests revealed promising outcomes but also highlighted key challenges: the models encountered difficulties in providing in-depth thematic understandings, showed suspectibility to misinformation, tended towards monotonic responses after an extended period of play, and struggled to offer detailed biographical information. Despite these limitations, the study's findings open up new research avenues at the crossroads of artificial intelligence, machine learning, LLMs, ethnography, anthropology and human-computer interaction.