EmoNet-Face: An Expert-Annotated Benchmark for Synthetic Emotion Recognition

TL;DR

EmoNet Face introduces a detailed emotion benchmark with a new taxonomy, diverse datasets, and expert annotations, enabling AI to better recognize nuanced human emotions and address biases in existing datasets.

Contribution

The paper presents EmoNet Face, a comprehensive benchmark with a novel emotion taxonomy, diverse controlled datasets, and a high-performance model, advancing emotion recognition research.

Findings

Achieved human-expert-level performance with EmpathicInsight-Face.

Created three large-scale, demographically balanced datasets.

Developed a 40-category emotion taxonomy grounded in research.

Abstract

Effective human-AI interaction relies on AI's ability to accurately perceive and interpret human emotions. Current benchmarks for vision and vision-language models are severely limited, offering a narrow emotional spectrum that overlooks nuanced states (e.g., bitterness, intoxication) and fails to distinguish subtle differences between related feelings (e.g., shame vs. embarrassment). Existing datasets also often use uncontrolled imagery with occluded faces and lack demographic diversity, risking significant bias. To address these critical gaps, we introduce EmoNet Face, a comprehensive benchmark suite. EmoNet Face features: (1) A novel 40-category emotion taxonomy, meticulously derived from foundational research to capture finer details of human emotional experiences. (2) Three large-scale, AI-generated datasets (EmoNet HQ, Binary, and Big) with explicit, full-face expressions and controlled demographic balance across ethnicity, age, and gender. (3) Rigorous, multi-expert annotations for training and high-fidelity evaluation. (4) We built EmpathicInsight-Face, a model achieving human-expert-level performance on our benchmark. The publicly released EmoNet Face suite - taxonomy, datasets, and model - provides a robust foundation for developing and evaluating AI systems with a deeper understanding of human emotions.