A Deep Features-Based Approach Using Modified ResNet50 and Gradient Boosting for Visual Sentiments Classification

TL;DR

This paper introduces a novel approach combining modified ResNet50 deep features with gradient boosting for improved visual sentiment classification, outperforming existing methods on benchmark datasets.

Contribution

It proposes a fusion of deep learning and machine learning techniques specifically for multiclass visual sentiment analysis, focusing on feature extraction and classification.

Findings

Outperforms state-of-the-art methods on CrowdFlower and GAPED datasets.

Effective deep feature extraction from modified ResNet50.

Gradient boosting enhances classification accuracy.

Abstract

The versatile nature of Visual Sentiment Analysis (VSA) is one reason for its rising profile. It isn't easy to efficiently manage social media data with visual information since previous research has concentrated on Sentiment Analysis (SA) of single modalities, like textual. In addition, most visual sentiment studies need to adequately classify sentiment because they are mainly focused on simply merging modal attributes without investigating their intricate relationships. This prompted the suggestion of developing a fusion of deep learning and machine learning algorithms. In this research, a deep feature-based method for multiclass classification has been used to extract deep features from modified ResNet50. Furthermore, gradient boosting algorithm has been used to classify photos containing emotional content. The approach is thoroughly evaluated on two benchmarked datasets, CrowdFlower and GAPED. Finally, cutting-edge deep learning and machine learning models were used to compare the proposed strategy. When compared to state-of-the-art approaches, the proposed method demonstrates exceptional performance on the datasets presented.