Toward Cultural Interpretability: A Linguistic Anthropological Framework for Describing and Evaluating Large Language Models (LLMs)

TL;DR

This paper introduces cultural interpretability, a framework combining linguistic anthropology and machine learning to analyze how large language models reflect and produce culturally sensitive language use.

Contribution

It proposes a new interdisciplinary approach, cultural interpretability, to study LLMs through the lens of linguistic anthropology, emphasizing cultural context in language model analysis.

Findings

Demonstrates the feasibility of analyzing LLMs through cultural interpretability.

Highlights how LLMs can reflect cultural relationships in language.

Suggests pathways for improving LLM alignment with diverse cultures.

Abstract

This article proposes a new integration of linguistic anthropology and machine learning (ML) around convergent interests in both the underpinnings of language and making language technologies more socially responsible. While linguistic anthropology focuses on interpreting the cultural basis for human language use, the ML field of interpretability is concerned with uncovering the patterns that Large Language Models (LLMs) learn from human verbal behavior. Through the analysis of a conversation between a human user and an LLM-powered chatbot, we demonstrate the theoretical feasibility of a new, conjoint field of inquiry, cultural interpretability (CI). By focusing attention on the communicative competence involved in the way human users and AI chatbots co-produce meaning in the articulatory interface of human-computer interaction, CI emphasizes how the dynamic relationship between language and culture makes contextually sensitive, open-ended conversation possible. We suggest that, by examining how LLMs internally "represent" relationships between language and culture, CI can: (1) provide insight into long-standing linguistic anthropological questions about the patterning of those relationships; and (2) aid model developers and interface designers in improving value alignment between language models and stylistically diverse speakers and culturally diverse speech communities. Our discussion proposes three critical research axes: relativity, variation, and indexicality.