Visual Sentiment Prediction with Deep Convolutional Neural Networks

TL;DR

This paper introduces a deep learning framework for predicting sentiment from images using transfer learning with CNNs, addressing the gap in visual sentiment analysis.

Contribution

It proposes a novel CNN-based framework that leverages transfer learning for visual sentiment prediction, validated on social media datasets.

Findings

Effective sentiment prediction on Twitter and Tumblr images

Transfer learning improves accuracy over traditional methods

Demonstrates feasibility of deep CNNs for visual sentiment analysis

Abstract

Images have become one of the most popular types of media through which users convey their emotions within online social networks. Although vast amount of research is devoted to sentiment analysis of textual data, there has been very limited work that focuses on analyzing sentiment of image data. In this work, we propose a novel visual sentiment prediction framework that performs image understanding with Deep Convolutional Neural Networks (CNN). Specifically, the proposed sentiment prediction framework performs transfer learning from a CNN with millions of parameters, which is pre-trained on large-scale data for object recognition. Experiments conducted on two real-world datasets from Twitter and Tumblr demonstrate the effectiveness of the proposed visual sentiment analysis framework.