Delving into Youth Perspectives on In-game Gambling-like Elements: A Proof-of-Concept Study Utilising Large Language Models for Analysing User-Generated Text Data

TL;DR

This study explores the use of Large Language Models for analyzing user opinions on gambling-like elements in digital games, demonstrating their effectiveness and highlighting areas for methodological improvement.

Contribution

It introduces a novel application of LLMs with prompting techniques for analyzing complex user-generated text data related to in-game gambling mechanics.

Findings

LLMs can identify relevant themes comparable to human coders

Prompt refinement improves analysis accuracy

Challenges remain in complex text interpretation

Abstract

This report documents the development, test, and application of Large Language Models (LLMs) for automated text analysis, with a specific focus on gambling-like elements in digital games, such as lootboxes. The project aimed not only to analyse user opinions and attitudes towards these mechanics, but also to advance methodological research in text analysis. By employing prompting techniques and iterative prompt refinement processes, the study sought to test and improve the accuracy of LLM-based text analysis. The findings indicate that while LLMs can effectively identify relevant patterns and themes on par with human coders, there are still challenges in handling more complex tasks, underscoring the need for ongoing refinement in methodologies. The methodological advancements achieved through this study significantly enhance the application of LLMs in real-world text analysis. The research provides valuable insights into how these models can be better utilized to analyze complex, user-generated content.