Meta-Cultural Competence: Climbing the Right Hill of Cultural Awareness

TL;DR

This paper argues that for LLMs to be truly useful across diverse cultures, they need meta-cultural competence rather than just cultural awareness, emphasizing principles and measurement methods.

Contribution

It introduces the concept of meta-cultural competence for LLMs, extending the Octopus test to define and evaluate this broader cultural capability.

Findings

Meta-cultural competence is essential for cross-cultural usefulness.

Proposes principles for measuring and modeling meta-cultural competence.

Extends the Octopus test to assess cultural capabilities.

Abstract

Numerous recent studies have shown that Large Language Models (LLMs) are biased towards a Western and Anglo-centric worldview, which compromises their usefulness in non-Western cultural settings. However, "culture" is a complex, multifaceted topic, and its awareness, representation, and modeling in LLMs and LLM-based applications can be defined and measured in numerous ways. In this position paper, we ask what does it mean for an LLM to possess "cultural awareness", and through a thought experiment, which is an extension of the Octopus test proposed by Bender and Koller (2020), we argue that it is not cultural awareness or knowledge, rather meta-cultural competence, which is required of an LLM and LLM-based AI system that will make it useful across various, including completely unseen, cultures. We lay out the principles of meta-cultural competence AI systems, and discuss ways to measure and model those.