Does Mapo Tofu Contain Coffee? Probing LLMs for Food-related Cultural Knowledge
Li Zhou, Taelin Karidi, Wanlong Liu, Nicolas Garneau, Yong Cao, Wenyu, Chen, Haizhou Li, Daniel Hershcovich

TL;DR
This paper investigates how large language models understand and exhibit biases in food-related cultural knowledge, introducing a multilingual dataset and analyzing factors influencing their cultural comprehension.
Contribution
It introduces FmLAMA, a multilingual dataset on food culture, and systematically evaluates LLMs' cultural knowledge across languages and architectures.
Findings
LLMs show bias towards US food knowledge
Cultural context improves LLM performance
Model effectiveness depends on language, architecture, and cultural nuances
Abstract
Recent studies have highlighted the presence of cultural biases in Large Language Models (LLMs), yet often lack a robust methodology to dissect these phenomena comprehensively. Our work aims to bridge this gap by delving into the Food domain, a universally relevant yet culturally diverse aspect of human life. We introduce FmLAMA, a multilingual dataset centered on food-related cultural facts and variations in food practices. We analyze LLMs across various architectures and configurations, evaluating their performance in both monolingual and multilingual settings. By leveraging templates in six different languages, we investigate how LLMs interact with language-specific and cultural knowledge. Our findings reveal that (1) LLMs demonstrate a pronounced bias towards food knowledge prevalent in the United States; (2) Incorporating relevant cultural context significantly improves LLMs'…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
Taxonomy
TopicsCulinary Culture and Tourism
