Decoding Visual Sentiment of Political Imagery

TL;DR

This paper presents a novel deep learning approach that incorporates ideological differences into visual sentiment analysis, improving accuracy in understanding political imagery from diverse perspectives.

Contribution

It introduces a multi-task model that accounts for attitudinal differences, addressing label ambiguity in visual sentiment classification of political images.

Findings

Improved sentiment prediction accuracy across ideological viewpoints

A new dataset reflecting partisan divides in visual sentiment labels

Demonstrated effectiveness on immigration-related images

Abstract

How can we define visual sentiment when viewers systematically disagree on their perspectives? This study introduces a novel approach to visual sentiment analysis by integrating attitudinal differences into visual sentiment classification. Recognizing that societal divides, such as partisan differences, heavily influence sentiment labeling, we developed a dataset that reflects these divides. We then trained a deep learning multi-task multi-class model to predict visual sentiment from different ideological viewpoints. Applied to immigration-related images, our approach captures perspectives from both Democrats and Republicans. By incorporating diverse perspectives into the labeling and model training process, our strategy addresses the limitation of label ambiguity and demonstrates improved accuracy in visual sentiment predictions. Overall, our study advocates for a paradigm shift in decoding visual sentiment toward creating classifiers that more accurately reflect the sentiments generated by humans.