Cross-Cultural and Cultural-Specific Production and Perception of Facial Expressions of Emotion in the Wild

TL;DR

This large-scale study investigates how facial expressions of emotion are produced and perceived across cultures in natural settings, revealing a limited set of expressions used for communication and differing perception patterns.

Contribution

It provides the first comprehensive analysis of cross-cultural and cultural-specific facial expressions of emotion in real-world conditions, highlighting their production, perception, and communication differences.

Findings

Only 35 of 16,384 possible configurations are used cross-culturally.

Cross-cultural expressions reliably convey emotion categories and valence.

Cultural-specific expressions reliably convey valence and arousal.

Abstract

Automatic recognition of emotion from facial expressions is an intense area of research, with a potentially long list of important application. Yet, the study of emotion requires knowing which facial expressions are used within and across cultures in the wild, not in controlled lab conditions; but such studies do not exist. Which and how many cross-cultural and cultural-specific facial expressions do people commonly use? And, what affect variables does each expression communicate to observers? If we are to design technology that understands the emotion of users, we need answers to these two fundamental questions. In this paper, we present the first large-scale study of the production and visual perception of facial expressions of emotion in the wild. We find that of the 16,384 possible facial configurations that people can theoretically produce, only 35 are successfully used to transmit emotive information across cultures, and only 8 within a smaller number of cultures. Crucially, we find that visual analysis of cross-cultural expressions yields consistent perception of emotion categories and valence, but not arousal. In contrast, visual analysis of cultural-specific expressions yields consistent perception of valence and arousal, but not of emotion categories. Additionally, we find that the number of expressions used to communicate each emotion is also different, e.g., 17 expressions transmit happiness, but only 1 is used to convey disgust.