Pragmatics Meets Culture: Culturally-adapted Artwork Description Generation and Evaluation
Lingjun Zhao, Dayeon Ki, Marine Carpuat, Hal Daum\'e III

TL;DR
This paper introduces culturally-adapted artwork description generation, evaluating models' cultural competence and demonstrating that pragmatic speaker models improve comprehension and helpfulness for diverse cultural audiences.
Contribution
It proposes a new task for culturally-adapted art description generation and a framework for evaluating cultural competence using grounded question answering.
Findings
Base models are only marginally adequate for cultural adaptation.
Pragmatic speaker models improve listener comprehension by up to 8.2%.
Human evaluations show increased helpfulness with higher pragmatic competence.
Abstract
Language models are known to exhibit various forms of cultural bias in decision-making tasks, yet much less is known about their degree of cultural familiarity in open-ended text generation tasks. In this paper, we introduce the task of culturally-adapted art description generation, where models describe artworks for audiences from different cultural groups who vary in their familiarity with the cultural symbols and narratives embedded in the artwork. To evaluate cultural competence in this pragmatic generation task, we propose a framework based on culturally grounded question answering. We find that base models are only marginally adequate for this task, but, through a pragmatic speaker model, we can improve simulated listener comprehension by up to 8.2%. A human study further confirms that the model with higher pragmatic competence is rated as more helpful for comprehension by 8.0%.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
