Building a Large Scale Dataset for Image Emotion Recognition: The Fine Print and The Benchmark

TL;DR

This paper introduces a large-scale, well-labeled dataset for image emotion recognition, significantly surpassing previous datasets in size, and provides benchmarking results using state-of-the-art CNN methods to advance research in visual emotion analysis.

Contribution

The paper presents a new large-scale dataset for image emotion recognition and offers extensive benchmarking with modern CNN techniques.

Findings

The dataset is 30 times larger than previous datasets.

State-of-the-art CNN models achieve promising results on this dataset.

The dataset facilitates further research in visual emotion analysis.

Abstract

Psychological research results have confirmed that people can have different emotional reactions to different visual stimuli. Several papers have been published on the problem of visual emotion analysis. In particular, attempts have been made to analyze and predict people's emotional reaction towards images. To this end, different kinds of hand-tuned features are proposed. The results reported on several carefully selected and labeled small image data sets have confirmed the promise of such features. While the recent successes of many computer vision related tasks are due to the adoption of Convolutional Neural Networks (CNNs), visual emotion analysis has not achieved the same level of success. This may be primarily due to the unavailability of confidently labeled and relatively large image data sets for visual emotion analysis. In this work, we introduce a new data set, which started from 3+ million weakly labeled images of different emotions and ended up 30 times as large as the current largest publicly available visual emotion data set. We hope that this data set encourages further research on visual emotion analysis. We also perform extensive benchmarking analyses on this large data set using the state of the art methods including CNNs.