Modeling Cross-Cultural Pragmatic Inference with Codenames Duet

TL;DR

This paper introduces the Cultural Codes dataset based on Codenames Duet to study how sociocultural backgrounds influence pragmatic inference in reference games, demonstrating that background knowledge improves predictive models.

Contribution

It creates a new dataset capturing sociocultural factors in a collaborative word game and shows their importance in modeling pragmatic reasoning.

Findings

Accounting for sociocultural priors improves model accuracy.

Sociocultural background significantly influences gameplay decisions.

The dataset includes diverse player demographics and personality data.

Abstract

Pragmatic reference enables efficient interpersonal communication. Prior work uses simple reference games to test models of pragmatic reasoning, often with unidentified speakers and listeners. In practice, however, speakers' sociocultural background shapes their pragmatic assumptions. For example, readers of this paper assume NLP refers to "Natural Language Processing," and not "Neuro-linguistic Programming." This work introduces the Cultural Codes dataset, which operationalizes sociocultural pragmatic inference in a simple word reference game. Cultural Codes is based on the multi-turn collaborative two-player game, Codenames Duet. Our dataset consists of 794 games with 7,703 turns, distributed across 153 unique players. Alongside gameplay, we collect information about players' personalities, values, and demographics. Utilizing theories of communication and pragmatics, we predict each player's actions via joint modeling of their sociocultural priors and the game context. Our experiments show that accounting for background characteristics significantly improves model performance for tasks related to both clue giving and guessing, indicating that sociocultural priors play a vital role in gameplay decisions.