Towards an "In-the-Wild" Emotion Dataset Using a Game-based Framework

TL;DR

This paper introduces a novel game-based framework for collecting a large, balanced in-the-wild facial emotion dataset called GaMo, leveraging deep learning for automatic labeling during gameplay.

Contribution

It presents a new game-based data collection method that automatically labels facial emotions, resulting in a sizable, balanced dataset for emotion recognition.

Findings

GaMo dataset contains over 15,000 images.

Models trained on GaMo outperform those trained on CIFE.

Game-based collection yields large, balanced emotion datasets.

Abstract

In order to create an "in-the-wild" dataset of facial emotions with large number of balanced samples, this paper proposes a game-based data collection framework. The framework mainly include three components: a game engine, a game interface, and a data collection and evaluation module. We use a deep learning approach to build an emotion classifier as the game engine. Then a emotion web game to allow gamers to enjoy the games, while the data collection module obtains automatically-labelled emotion images. Using our game, we have collected more than 15,000 images within a month of the test run and built an emotion dataset "GaMo". To evaluate the dataset, we compared the performance of two deep learning models trained on both GaMo and CIFE. The results of our experiments show that because of being large and balanced, GaMo can be used to build a more robust emotion detector than the emotion detector trained on CIFE, which was used in the game engine to collect the face images.