Sentiment Classification using Images and Label Embeddings

TL;DR

This paper investigates the role of images in sentiment analysis by comparing models using only images, only text, and combined data, to assess the added value and generalization capabilities of image data.

Contribution

It introduces a comparative analysis of image-only, text-only, and combined models for sentiment classification, highlighting the impact of images on performance and generalization.

Findings

Images carry significant semantic information for sentiment analysis.

Combining images with text improves classification accuracy.

The approach enhances model generalization to unseen sentiments.

Abstract

In this project we analysed how much semantic information images carry, and how much value image data can add to sentiment analysis of the text associated with the images. To better understand the contribution from images, we compared models which only made use of image data, models which only made use of text data, and models which combined both data types. We also analysed if this approach could help sentiment classifiers generalize to unknown sentiments.