EmotioNet Challenge: Recognition of facial expressions of emotion in the wild

TL;DR

The EmotioNet challenge evaluated computer vision algorithms on facial expression analysis in unconstrained settings, revealing current limitations especially in emotion recognition and highlighting key factors affecting performance.

Contribution

This paper introduces the first large-scale challenge for automatic facial expression analysis in the wild, benchmarking current algorithms and identifying key limitations.

Findings

Current algorithms struggle with reliable emotion recognition.

3D pose significantly impacts facial expression analysis.

Algorithms are robust to mild resolution, occlusion, gender, and age variations.

Abstract

This paper details the methodology and results of the EmotioNet challenge. This challenge is the first to test the ability of computer vision algorithms in the automatic analysis of a large number of images of facial expressions of emotion in the wild. The challenge was divided into two tracks. The first track tested the ability of current computer vision algorithms in the automatic detection of action units (AUs). Specifically, we tested the detection of 11 AUs. The second track tested the algorithms' ability to recognize emotion categories in images of facial expressions. Specifically, we tested the recognition of 16 basic and compound emotion categories. The results of the challenge suggest that current computer vision and machine learning algorithms are unable to reliably solve these two tasks. The limitations of current algorithms are more apparent when trying to recognize emotion. We also show that current algorithms are not affected by mild resolution changes, small occluders, gender or age, but that 3D pose is a major limiting factor on performance. We provide an in-depth discussion of the points that need special attention moving forward.