Shades of smiles: creating variants of smiles from neutral images of real individuals - method and validation

TL;DR

This paper introduces a method to generate subtle facial expressions like reward, affiliative, and dominance smiles from neutral photos, validated through ratings on emotion and plausibility.

Contribution

A novel pipeline for generating fine-grained emotional expressions using FACS principles and validation of the stimuli through systematic ratings.

Findings

Reward smiles were most reliably recognized, while affiliative and dominance smiles were often confused with other expressions.

Arousal and plausibility ratings varied systematically across expression types and model gender.

The pipeline and dataset provide a scalable resource for emotion recognition and communication research.

Abstract

Despite their crucial role in emotion research, facial expression databases primarily contain stimuli of basic emotions, rather than subtle, socially nuanced ones. Here, we introduce a rigorous yet easily applicable pipeline for synthesizing fine-grained emotional expressions—specifically, reward, affiliative, and dominance smiles—from neutral photographs, guided by Facial Action Coding System (FACS) principles and the perspective that expressions serve as social signals. From neutral images of 90 individuals (45 female, 45 male), we generated five expressions each (including neutral and disgust), and mirrored each image to address hemiface biases. In an online validation study, 13 participants rated each image on emotional content, arousal, and plausibility, yielding 26 ratings per expression and model. Rating results show that (1) reward smiles were most reliably recognized, while affiliative and dominance smiles tended to be confused with related expressions and that (2) arousal and plausibility ratings varied systematically across expression types and model gender. In summary, the suggested expression generation pipeline and the validated stimuli offer a robust method and a scalable dataset for research on fine-grained emotion recognition and emotional communication.

Figures