MOSI: Multimodal Corpus of Sentiment Intensity and Subjectivity Analysis in Online Opinion Videos

TL;DR

This paper introduces MOSI, a comprehensive multimodal dataset with annotations for sentiment and subjectivity in online opinion videos, along with baseline models and a fusion approach for multimodal analysis.

Contribution

It provides the first opinion-level annotated multimodal dataset for sentiment analysis in videos, including visual and audio features, and proposes a new fusion method.

Findings

Dataset includes detailed annotations for sentiment, subjectivity, visual, and audio features.

Baseline models demonstrate the potential of multimodal fusion for sentiment analysis.

A novel multimodal fusion approach effectively combines spoken words and visual gestures.

Abstract

People are sharing their opinions, stories and reviews through online video sharing websites every day. Studying sentiment and subjectivity in these opinion videos is experiencing a growing attention from academia and industry. While sentiment analysis has been successful for text, it is an understudied research question for videos and multimedia content. The biggest setbacks for studies in this direction are lack of a proper dataset, methodology, baselines and statistical analysis of how information from different modality sources relate to each other. This paper introduces to the scientific community the first opinion-level annotated corpus of sentiment and subjectivity analysis in online videos called Multimodal Opinion-level Sentiment Intensity dataset (MOSI). The dataset is rigorously annotated with labels for subjectivity, sentiment intensity, per-frame and per-opinion annotated visual features, and per-milliseconds annotated audio features. Furthermore, we present baselines for future studies in this direction as well as a new multimodal fusion approach that jointly models spoken words and visual gestures.