A Multimodal Sentiment Dataset for Video Recommendation

TL;DR

This paper introduces DuVideoSenti, a new multimodal sentiment dataset tailored for video recommendation scenarios, featuring a novel sentiment system and serving as a benchmark for multimodal analysis and video understanding.

Contribution

The paper presents DuVideoSenti, a large-scale multimodal sentiment dataset with a new sentiment system for videos in recommendation contexts, and establishes baseline results using UNIMO.

Findings

DuVideoSenti contains 5,630 videos with sentiment labels.

The dataset introduces a new sentiment system for videos.

Baseline experiments with UNIMO highlight the dataset's challenges.

Abstract

Recently, multimodal sentiment analysis has seen remarkable advance and a lot of datasets are proposed for its development. In general, current multimodal sentiment analysis datasets usually follow the traditional system of sentiment/emotion, such as positive, negative and so on. However, when applied in the scenario of video recommendation, the traditional sentiment/emotion system is hard to be leveraged to represent different contents of videos in the perspective of visual senses and language understanding. Based on this, we propose a multimodal sentiment analysis dataset, named baiDu Video Sentiment dataset (DuVideoSenti), and introduce a new sentiment system which is designed to describe the sentimental style of a video on recommendation scenery. Specifically, DuVideoSenti consists of 5,630 videos which displayed on Baidu, each video is manually annotated with a sentimental style label which describes the user's real feeling of a video. Furthermore, we propose UNIMO as our baseline for DuVideoSenti. Experimental results show that DuVideoSenti brings new challenges to multimodal sentiment analysis, and could be used as a new benchmark for evaluating approaches designed for video understanding and multimodal fusion. We also expect our proposed DuVideoSenti could further improve the development of multimodal sentiment analysis and its application to video recommendations.