Strategic Insights in Human and Large Language Model Tactics at Word Guessing Games

TL;DR

This paper analyzes human strategies and the capabilities of large language models in a popular word-guessing game, revealing insights into human motivations and the models' limitations over two years.

Contribution

It provides a comprehensive study of human tactics and evaluates LLM performance in a real-world game context across multiple languages.

Findings

Humans develop evolving strategies and motivations.

Large language models struggle with guess length and repetitions.

Models exhibit hallucinations and inflections in gameplay.

Abstract

At the beginning of 2022, a simplistic word-guessing game took the world by storm and was further adapted to many languages beyond the original English version. In this paper, we examine the strategies of daily word-guessing game players that have evolved during a period of over two years. A survey gathered from 25% of frequent players reveals their strategies and motivations for continuing the daily journey. We also explore the capability of several popular open-access large language model systems and open-source models at comprehending and playing the game in two different languages. Results highlight the struggles of certain models to maintain correct guess length and generate repetitions, as well as hallucinations of non-existent words and inflections.