When Saliency Meets Sentiment: Understanding How Image Content Invokes Emotion and Sentiment

TL;DR

This paper explores how visual saliency influences emotional responses in images by analyzing the relationship between salient regions and overall sentiment across various scene types, providing insights into the mechanisms of visual sentiment evocation.

Contribution

It introduces a detailed analysis of the interaction between visual saliency and sentiment across multiple scene categories using advanced detection and classification algorithms.

Findings

Salient regions significantly influence overall image sentiment.

Different scene types show varied correlations between saliency and sentiment.

Insights shed light on how visual content evokes emotions in viewers.

Abstract

Sentiment analysis is crucial for extracting social signals from social media content. Due to the prevalence of images in social media, image sentiment analysis is receiving increasing attention in recent years. However, most existing systems are black-boxes that do not provide insight on how image content invokes sentiment and emotion in the viewers. Psychological studies have confirmed that salient objects in an image often invoke emotions. In this work, we investigate more fine-grained and more comprehensive interaction between visual saliency and visual sentiment. In particular, we partition images in several primary scene-type dimensions, including: open-closed, natural-manmade, indoor-outdoor, and face-noface. Using state of the art saliency detection algorithm and sentiment classification algorithm, we examine how the sentiment of the salient region(s) in an image relates to the overall sentiment of the image. The experiments on a representative image emotion dataset have shown interesting correlation between saliency and sentiment in different scene types and in turn shed light on the mechanism of visual sentiment evocation.