Affective Image Content Analysis: Two Decades Review and New Perspectives

TL;DR

This comprehensive review covers two decades of affective image content analysis, highlighting key methods, challenges like the affective gap and label noise, and future research directions in emotion recognition from images.

Contribution

It provides a thorough overview of AICA development, comparing state-of-the-art methods, datasets, and addressing challenges like subjectivity and noisy labels.

Findings

Deep features outperform handcrafted features in emotion recognition

Personalized emotion prediction improves accuracy over generic models

Future directions include understanding image context and viewer interactions

Abstract

Images can convey rich semantics and induce various emotions in viewers. Recently, with the rapid advancement of emotional intelligence and the explosive growth of visual data, extensive research efforts have been dedicated to affective image content analysis (AICA). In this survey, we will comprehensively review the development of AICA in the recent two decades, especially focusing on the state-of-the-art methods with respect to three main challenges -- the affective gap, perception subjectivity, and label noise and absence. We begin with an introduction to the key emotion representation models that have been widely employed in AICA and description of available datasets for performing evaluation with quantitative comparison of label noise and dataset bias. We then summarize and compare the representative approaches on (1) emotion feature extraction, including both handcrafted and deep features, (2) learning methods on dominant emotion recognition, personalized emotion prediction, emotion distribution learning, and learning from noisy data or few labels, and (3) AICA based applications. Finally, we discuss some challenges and promising research directions in the future, such as image content and context understanding, group emotion clustering, and viewer-image interaction.